[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:16:51 dist-test-slave-t3q3 krb5kdc[12256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:16:51 dist-test-slave-t3q3 krb5kdc[12256](info): set up 2 sockets
Aug 06 16:16:51 dist-test-slave-t3q3 krb5kdc[12256](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 16:16:52.544584 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.004s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:16:52 dist-test-slave-t3q3 krb5kdc[12256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:16:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:16:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:16:52.705163 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:34617
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:34617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:16:52.853963 12280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:16:52.854342 12280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:16:52.854387 12280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:16:52.858698 12280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:16:52.858798 12280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:16:52.858824 12280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:16:52.858842 12280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:16:52.858858 12280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:16:52.865208 12280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:34617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:34617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:16:52.866868 12280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:16:52.868378 12280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:16:52.879693 12287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:52.879683 12286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:16:52.879757 12280 server_base.cc:1034] running on GCE node
W20260806 16:16:52.879693 12292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:16:52.880873 12280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:16:52.882339 12280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:16:52.883515 12280 hybrid_clock.cc:648] HybridClock initialized: now 1786033012883520 us; error 97 us; skew 500 ppm
Aug 06 16:16:52 dist-test-slave-t3q3 krb5kdc[12256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033012, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:16:52.891577 12280 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:16:52.892968 12280 webserver.cc:460] Webserver started at http://127.11.245.190:37805/ using document root <none> and password file <none>
I20260806 16:16:52.893738 12280 fs_manager.cc:359] Metadata directory not provided
I20260806 16:16:52.893827 12280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:16:52.894068 12280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:16:52.897713 12280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "12d24ee33b2548bfbc3df39d3773bb14"
format_stamp: "Formatted at 2026-08-06 16:16:52 on dist-test-slave-t3q3"
server_key: "f84d8f5c813470fe823fe8fc371abdf9"
server_key_iv: "971080a644b1bbb75ce159280d2e4ae5"
server_key_version: "encryptionkey@0"
I20260806 16:16:52.898733 12280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "12d24ee33b2548bfbc3df39d3773bb14"
format_stamp: "Formatted at 2026-08-06 16:16:52 on dist-test-slave-t3q3"
server_key: "f84d8f5c813470fe823fe8fc371abdf9"
server_key_iv: "971080a644b1bbb75ce159280d2e4ae5"
server_key_version: "encryptionkey@0"
I20260806 16:16:52.904847 12280 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 16:16:52.908352 12303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:52.910095 12280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:16:52.910295 12280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "12d24ee33b2548bfbc3df39d3773bb14"
format_stamp: "Formatted at 2026-08-06 16:16:52 on dist-test-slave-t3q3"
server_key: "f84d8f5c813470fe823fe8fc371abdf9"
server_key_iv: "971080a644b1bbb75ce159280d2e4ae5"
server_key_version: "encryptionkey@0"
I20260806 16:16:52.910454 12280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:16:52.925967 12280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:16:52.930370 12280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:16:52.930645 12280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:16:52.942819 12280 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:34617
I20260806 16:16:52.942879 12384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:34617 every 8 connection(s)
I20260806 16:16:52.944518 12280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:16:52.947003 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 12280
I20260806 16:16:52.947249 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:16:52.948961 12385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:16:52.949167 12246 external_mini_cluster.cc:1428] Setting key d267a576ab1e5ad4a815c2d61d3097d3
I20260806 16:16:52.957587 12385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14: Bootstrap starting.
I20260806 16:16:52.960986 12385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14: Neither blocks nor log segments found. Creating new log.
I20260806 16:16:52.961963 12385 log.cc:824] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14: Log is configured to *not* fsync() on all Append() calls
I20260806 16:16:52.965005 12385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14: No bootstrap required, opened a new log
I20260806 16:16:52.969513 12385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d24ee33b2548bfbc3df39d3773bb14" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34617 } }
I20260806 16:16:52.969852 12385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:16:52.969949 12385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12d24ee33b2548bfbc3df39d3773bb14, State: Initialized, Role: FOLLOWER
I20260806 16:16:52.970533 12385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d24ee33b2548bfbc3df39d3773bb14" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34617 } }
I20260806 16:16:52.970727 12385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:16:52.970816 12385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:16:52.970963 12385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 16:16:52 dist-test-slave-t3q3 krb5kdc[12256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:16:52.972234 12385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d24ee33b2548bfbc3df39d3773bb14" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34617 } }
I20260806 16:16:52.972694 12385 leader_election.cc:302] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12d24ee33b2548bfbc3df39d3773bb14; no voters:
I20260806 16:16:52.973189 12385 leader_election.cc:288] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:16:52.973368 12392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:16:52.974038 12392 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [term 1 LEADER]: Becoming Leader. State: Replica: 12d24ee33b2548bfbc3df39d3773bb14, State: Running, Role: LEADER
I20260806 16:16:52.974697 12392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d24ee33b2548bfbc3df39d3773bb14" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34617 } }
I20260806 16:16:52.975165 12385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:16:52.977174 12393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12d24ee33b2548bfbc3df39d3773bb14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d24ee33b2548bfbc3df39d3773bb14" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34617 } } }
I20260806 16:16:52.977739 12393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [sys.catalog]: This master's current role is: LEADER
I20260806 16:16:52.978504 12395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12d24ee33b2548bfbc3df39d3773bb14. Latest consensus state: current_term: 1 leader_uuid: "12d24ee33b2548bfbc3df39d3773bb14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d24ee33b2548bfbc3df39d3773bb14" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34617 } } }
I20260806 16:16:52.978638 12395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14 [sys.catalog]: This master's current role is: LEADER
W20260806 16:16:52.982674 12417 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:16:52.982770 12417 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:16:52.982789 12387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:52.953898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55406 (local address 127.11.245.190:34617)
0806 16:16:52.954517 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:52.954539 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:52.955759 (+ 1220us) server_negotiation.cc:407] Connection header received
0806 16:16:52.959957 (+ 4198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:52.959974 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:52.960463 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:52.960847 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:52.963517 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:52.965129 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:52.966462 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:52.967436 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:52.967875 (+ 439us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:52.973082 (+ 5207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:52.973095 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:52.973120 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:52.973169 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:52.975519 (+ 2350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:52.976404 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:52.976412 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:52.976427 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:52.976605 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:52.978368 (+ 1763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:52.978375 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:52.978381 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:52.979983 (+ 1602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:52.980346 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:52.981238 (+ 892us) server_negotiation.cc:299] Negotiation successful
0806 16:16:52.981609 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":198,"threads_started":1}
I20260806 16:16:52.982889 12413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:16:52.983717 12413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:16:52.991047 12413 catalog_manager.cc:1347] Generated new cluster ID: f55032a07fe948fc88496a9c9f067382
I20260806 16:16:52.991171 12413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:16:53.007237 12413 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:16:53.008934 12413 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:16:53.014627 12413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12d24ee33b2548bfbc3df39d3773bb14: Generated new TSK 0
I20260806 16:16:53.015774 12413 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:16:53.065546 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34617
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:16:53.224406 12431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:16:53.224790 12431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:16:53.224852 12431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:16:53.229511 12431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:16:53.229598 12431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:16:53.229708 12431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:16:53.235960 12431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34617
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:16:53.237684 12431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:16:53.239100 12431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:16:53.252370 12440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:53.252413 12441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:16:53.252463 12431 server_base.cc:1034] running on GCE node
W20260806 16:16:53.252386 12446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:16:53.253615 12431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:16:53.254647 12431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:16:53.255851 12431 hybrid_clock.cc:648] HybridClock initialized: now 1786033013255830 us; error 39 us; skew 500 ppm
Aug 06 16:16:53 dist-test-slave-t3q3 krb5kdc[12256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033013, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:16:53.259709 12431 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:16:53.260993 12431 webserver.cc:460] Webserver started at http://127.11.245.129:37615/ using document root <none> and password file <none>
I20260806 16:16:53.261771 12431 fs_manager.cc:359] Metadata directory not provided
I20260806 16:16:53.261878 12431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:16:53.262177 12431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:16:53.264963 12431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "ca5591ffe95a4a799f612ff7021b53c8"
format_stamp: "Formatted at 2026-08-06 16:16:53 on dist-test-slave-t3q3"
server_key: "7e77330786d9ff90641f91b0b3bb3a78"
server_key_iv: "37762b9981472edce97572a0e96a493d"
server_key_version: "encryptionkey@0"
I20260806 16:16:53.265702 12431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "ca5591ffe95a4a799f612ff7021b53c8"
format_stamp: "Formatted at 2026-08-06 16:16:53 on dist-test-slave-t3q3"
server_key: "7e77330786d9ff90641f91b0b3bb3a78"
server_key_iv: "37762b9981472edce97572a0e96a493d"
server_key_version: "encryptionkey@0"
I20260806 16:16:53.270895 12431 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:16:53.274464 12455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:53.276746 12431 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:16:53.276938 12431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "ca5591ffe95a4a799f612ff7021b53c8"
format_stamp: "Formatted at 2026-08-06 16:16:53 on dist-test-slave-t3q3"
server_key: "7e77330786d9ff90641f91b0b3bb3a78"
server_key_iv: "37762b9981472edce97572a0e96a493d"
server_key_version: "encryptionkey@0"
I20260806 16:16:53.277122 12431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:16:53.290923 12431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:16:53.296162 12431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:16:53.296449 12431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:16:53.297712 12431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:16:53.299108 12431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:16:53.299193 12431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:53.299494 12431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:16:53.299590 12431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:53.312433 12431 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:38443
I20260806 16:16:53.312467 12640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:38443 every 8 connection(s)
I20260806 16:16:53.313687 12431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:16:53.318321 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 12431
I20260806 16:16:53.318465 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:16:53.318799 12246 external_mini_cluster.cc:1428] Setting key 545d192dacf3d5ba4e35bb9a99911052
Aug 06 16:16:53 dist-test-slave-t3q3 krb5kdc[12256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033013, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:16:53.333796 12387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:53.316287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:54833 (local address 127.11.245.190:34617)
0806 16:16:53.316485 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:53.316490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:53.318076 (+ 1586us) server_negotiation.cc:407] Connection header received
0806 16:16:53.319433 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:53.319436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:53.319509 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:53.319788 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:53.321672 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:53.322685 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:53.324195 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:53.324467 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:53.324593 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:53.328050 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:53.328055 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:53.328057 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:53.328148 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:53.330302 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:53.331140 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:53.331147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:53.331152 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:53.331272 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:53.331889 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:53.331893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:53.331896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:53.332258 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:53.332374 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:53.333298 (+ 924us) server_negotiation.cc:299] Negotiation successful
0806 16:16:53.333508 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:16:53.334867 12643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:53.316880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:34617 (local address 127.11.245.129:54833)
0806 16:16:53.317879 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:53.317933 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:53.319123 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:53.319892 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:53.319901 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:53.320643 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:16:53.321474 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:53.321495 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:53.322865 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:53.322869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:53.323986 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:53.323999 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:53.324668 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:53.324672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:53.324880 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:53.325665 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:16:53.325686 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:16:53.327783 (+ 2097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:16:53.330605 (+ 2822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:53.330613 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:53.330634 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:53.330998 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:53.331546 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:53.331549 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:53.331551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:53.331756 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:53.332546 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:16:53.332554 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:16:53.332932 (+ 378us) client_negotiation.cc:769] Sending connection context
0806 16:16:53.333172 (+ 240us) client_negotiation.cc:240] Negotiation successful
0806 16:16:53.333574 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":149,"threads_started":1}
I20260806 16:16:53.336790 12641 heartbeater.cc:347] Connected to a master server at 127.11.245.190:34617
I20260806 16:16:53.337134 12641 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
I20260806 16:16:53.337942 12641 heartbeater.cc:511] Master 127.11.245.190:34617 requested a full tablet report, sending...
I20260806 16:16:53.340909 12335 ts_manager.cc:194] Registered new tserver with Master: ca5591ffe95a4a799f612ff7021b53c8 (127.11.245.129:38443)
I20260806 16:16:53.342566 12335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:54833
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:16:53.360051 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34617
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:16:53.511395 12646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:16:53.511974 12646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:16:53.512105 12646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:16:53.517288 12646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:16:53.517400 12646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:16:53.517503 12646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:16:53.523872 12646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34617
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:16:53.525523 12646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:16:53.527068 12646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:16:53.539855 12655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:53.539842 12662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:53.539898 12660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:16:53.540422 12646 server_base.cc:1034] running on GCE node
I20260806 16:16:53.541015 12646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:16:53.541870 12646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:16:53.543087 12646 hybrid_clock.cc:648] HybridClock initialized: now 1786033013543057 us; error 36 us; skew 500 ppm
Aug 06 16:16:53 dist-test-slave-t3q3 krb5kdc[12256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033013, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:16:53.547488 12646 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:16:53.549220 12646 webserver.cc:460] Webserver started at http://127.11.245.130:42013/ using document root <none> and password file <none>
I20260806 16:16:53.550293 12646 fs_manager.cc:359] Metadata directory not provided
I20260806 16:16:53.550426 12646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:16:53.550793 12646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:16:53.554930 12646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "62bdca3430b340759997e2ca27d79446"
format_stamp: "Formatted at 2026-08-06 16:16:53 on dist-test-slave-t3q3"
server_key: "32ba641c7221bff8a9d3ade73754e338"
server_key_iv: "9cf224b5f72d660060ab5368178b180f"
server_key_version: "encryptionkey@0"
I20260806 16:16:53.556129 12646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "62bdca3430b340759997e2ca27d79446"
format_stamp: "Formatted at 2026-08-06 16:16:53 on dist-test-slave-t3q3"
server_key: "32ba641c7221bff8a9d3ade73754e338"
server_key_iv: "9cf224b5f72d660060ab5368178b180f"
server_key_version: "encryptionkey@0"
I20260806 16:16:53.563192 12646 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 16:16:53.567502 12670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:53.569365 12646 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 16:16:53.569504 12646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "62bdca3430b340759997e2ca27d79446"
format_stamp: "Formatted at 2026-08-06 16:16:53 on dist-test-slave-t3q3"
server_key: "32ba641c7221bff8a9d3ade73754e338"
server_key_iv: "9cf224b5f72d660060ab5368178b180f"
server_key_version: "encryptionkey@0"
I20260806 16:16:53.569634 12646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:16:53.599185 12646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:16:53.604907 12646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:16:53.605134 12646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:16:53.605958 12646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:16:53.607085 12646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:16:53.607139 12646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:53.607190 12646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:16:53.607209 12646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:53.619628 12646 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:41273
I20260806 16:16:53.619663 12851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:41273 every 8 connection(s)
I20260806 16:16:53.621007 12646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:16:53.622328 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 12646
I20260806 16:16:53.622467 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:16:53.622809 12246 external_mini_cluster.cc:1428] Setting key 18904e36580b95d283f987cd1d7ec912
Aug 06 16:16:53 dist-test-slave-t3q3 krb5kdc[12256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033013, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
I20260806 16:16:53.642607 12387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:53.624172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:50541 (local address 127.11.245.190:34617)
0806 16:16:53.624472 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:53.624478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:53.626530 (+ 2052us) server_negotiation.cc:407] Connection header received
0806 16:16:53.627759 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:53.627763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:53.627832 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:53.627929 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:53.629621 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:53.630229 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:53.632305 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:53.632594 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:53.632703 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:53.636834 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:53.636858 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:53.636863 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:53.636905 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:53.638892 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:53.639715 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:53.639720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:53.639722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:53.639831 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:53.640808 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:53.640822 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:53.640827 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:53.641130 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:53.641231 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:53.642210 (+ 979us) server_negotiation.cc:299] Negotiation successful
0806 16:16:53.642400 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260806 16:16:53.643877 12858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:53.625300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:34617 (local address 127.11.245.130:50541)
0806 16:16:53.626333 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:53.626404 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:53.627490 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:53.628232 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:53.628243 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:53.628743 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:16:53.629435 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:53.629453 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:53.630393 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:53.630400 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:53.632099 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:53.632119 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:53.632821 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:53.632828 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:53.633261 (+ 433us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:53.634138 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:16:53.634169 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:16:53.636483 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:16:53.639072 (+ 2589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:53.639080 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:53.639100 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:53.639571 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:53.640121 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:53.640128 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:53.640130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:53.640526 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:53.641430 (+ 904us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:16:53.641438 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:16:53.641871 (+ 433us) client_negotiation.cc:769] Sending connection context
0806 16:16:53.642146 (+ 275us) client_negotiation.cc:240] Negotiation successful
0806 16:16:53.642629 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"thread_start_us":213,"threads_started":1}
I20260806 16:16:53.645707 12853 heartbeater.cc:347] Connected to a master server at 127.11.245.190:34617
I20260806 16:16:53.646082 12853 heartbeater.cc:464] Registering TS with master...
I20260806 16:16:53.646847 12853 heartbeater.cc:511] Master 127.11.245.190:34617 requested a full tablet report, sending...
I20260806 16:16:53.648361 12335 ts_manager.cc:194] Registered new tserver with Master: 62bdca3430b340759997e2ca27d79446 (127.11.245.130:41273)
I20260806 16:16:53.649087 12335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:50541
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:16:53.664366 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34617
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:16:53.807489 12862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:16:53.807812 12862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:16:53.807868 12862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:16:53.811928 12862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:16:53.812000 12862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:16:53.812081 12862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:16:53.818257 12862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34617
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:16:53.819756 12862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:16:53.821180 12862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:16:53.833261 12868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:53.833222 12872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:53.833243 12867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:16:53.833597 12862 server_base.cc:1034] running on GCE node
I20260806 16:16:53.834177 12862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:16:53.834959 12862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:16:53.836156 12862 hybrid_clock.cc:648] HybridClock initialized: now 1786033013836130 us; error 34 us; skew 500 ppm
Aug 06 16:16:53 dist-test-slave-t3q3 krb5kdc[12256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033013, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:16:53.839545 12862 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:16:53.840694 12862 webserver.cc:460] Webserver started at http://127.11.245.131:42157/ using document root <none> and password file <none>
I20260806 16:16:53.841434 12862 fs_manager.cc:359] Metadata directory not provided
I20260806 16:16:53.841552 12862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:16:53.841828 12862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:16:53.844566 12862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "afae4ca59236439aad5ff424d2845b09"
format_stamp: "Formatted at 2026-08-06 16:16:53 on dist-test-slave-t3q3"
server_key: "efa9fd70ab8e6abf81a6904963fa962f"
server_key_iv: "e138d4542b4e2210cf3bf874b179e0b8"
server_key_version: "encryptionkey@0"
I20260806 16:16:53.845258 12862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "afae4ca59236439aad5ff424d2845b09"
format_stamp: "Formatted at 2026-08-06 16:16:53 on dist-test-slave-t3q3"
server_key: "efa9fd70ab8e6abf81a6904963fa962f"
server_key_iv: "e138d4542b4e2210cf3bf874b179e0b8"
server_key_version: "encryptionkey@0"
I20260806 16:16:53.850901 12862 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:16:53.854352 12882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:53.856472 12862 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:16:53.856648 12862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "afae4ca59236439aad5ff424d2845b09"
format_stamp: "Formatted at 2026-08-06 16:16:53 on dist-test-slave-t3q3"
server_key: "efa9fd70ab8e6abf81a6904963fa962f"
server_key_iv: "e138d4542b4e2210cf3bf874b179e0b8"
server_key_version: "encryptionkey@0"
I20260806 16:16:53.856817 12862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:16:53.875022 12862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:16:53.879464 12862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:16:53.879664 12862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:16:53.880435 12862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:16:53.881495 12862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:16:53.881580 12862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:53.881651 12862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:16:53.881685 12862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:53.893371 12862 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:41749
I20260806 16:16:53.893405 13059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:41749 every 8 connection(s)
I20260806 16:16:53.894572 12862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:16:53.895200 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 12862
I20260806 16:16:53.895340 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:16:53.895720 12246 external_mini_cluster.cc:1428] Setting key c583d75a81a44095ab8cba6349d0bc05
Aug 06 16:16:53 dist-test-slave-t3q3 krb5kdc[12256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033013, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:16:53.913751 12387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:53.897464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:36955 (local address 127.11.245.190:34617)
0806 16:16:53.899113 (+ 1649us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:53.899119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:53.899277 (+ 158us) server_negotiation.cc:407] Connection header received
0806 16:16:53.900516 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:53.900519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:53.900576 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:53.900654 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:53.902532 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:53.903266 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:53.905145 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:53.905400 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:53.905532 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:53.909143 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:53.909146 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:53.909149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:53.909180 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:53.911079 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:53.911797 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:53.911801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:53.911803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:53.911914 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:53.912392 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:53.912395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:53.912397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:53.912623 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:53.912708 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:53.913451 (+ 743us) server_negotiation.cc:299] Negotiation successful
0806 16:16:53.913587 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1544}
I20260806 16:16:53.914995 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:53.898073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:34617 (local address 127.11.245.131:36955)
0806 16:16:53.899088 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:53.899156 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:53.900267 (+ 1111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:53.900875 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:53.900883 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:53.901643 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:16:53.902309 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:53.902327 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:53.903498 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:53.903501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:53.904981 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:53.904995 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:53.905720 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:53.905730 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:53.906169 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:53.906970 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:16:53.906990 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:16:53.908912 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:16:53.911221 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:53.911231 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:53.911261 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:53.911664 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:53.912034 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:53.912037 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:53.912039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:53.912261 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:53.912743 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:16:53.912750 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:16:53.913129 (+ 379us) client_negotiation.cc:769] Sending connection context
0806 16:16:53.913344 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 16:16:53.913704 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":183,"threads_started":1}
I20260806 16:16:53.916589 13060 heartbeater.cc:347] Connected to a master server at 127.11.245.190:34617
I20260806 16:16:53.916949 13060 heartbeater.cc:464] Registering TS with master...
I20260806 16:16:53.917641 13060 heartbeater.cc:511] Master 127.11.245.190:34617 requested a full tablet report, sending...
I20260806 16:16:53.919001 12335 ts_manager.cc:194] Registered new tserver with Master: afae4ca59236439aad5ff424d2845b09 (127.11.245.131:41749)
I20260806 16:16:53.919723 12335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:36955
I20260806 16:16:53.924686 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 16:16:53 dist-test-slave-t3q3 krb5kdc[12256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033013, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 16:16:53 dist-test-slave-t3q3 krb5kdc[12256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033013, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.245.129@KRBTEST.COM
I20260806 16:16:53.960897 13074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:53.947777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43288 (local address 127.11.245.129:38443)
0806 16:16:53.948244 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:53.948255 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:53.948292 (+ 37us) server_negotiation.cc:407] Connection header received
0806 16:16:53.948398 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:53.948406 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:53.948730 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:53.948921 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:53.949788 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:53.951153 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:53.952215 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:53.952843 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:53.952962 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:53.956438 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:53.956452 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:53.956476 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:53.956524 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:53.958569 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:53.959090 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:53.959096 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:53.959097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:53.959189 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:53.959496 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:53.959499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:53.959500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:53.959805 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:53.959986 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:53.960374 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 16:16:53.960559 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":136,"threads_started":1}
W20260806 16:16:53.962352 12500 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43288
Aug 06 16:16:53 dist-test-slave-t3q3 krb5kdc[12256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 16:16:53 dist-test-slave-t3q3 krb5kdc[12256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.129@KRBTEST.COM
I20260806 16:16:53.998046 13074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:53.987500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43304 (local address 127.11.245.129:38443)
0806 16:16:53.987753 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:53.987757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:53.987881 (+ 124us) server_negotiation.cc:407] Connection header received
0806 16:16:53.988185 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:53.988189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:53.988255 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:53.988346 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:53.989289 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:53.989963 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:53.990984 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:53.991237 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:53.991338 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:53.994461 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:53.994465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:53.994468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:53.994503 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:53.996244 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:53.996731 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:53.996734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:53.996736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:53.996818 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:53.997067 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:53.997069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:53.997070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:53.997274 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:53.997389 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:53.997687 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 16:16:53.997814 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260806 16:16:53.999567 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 12431
I20260806 16:16:54.025102 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 12646
I20260806 16:16:54.057180 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 12862
I20260806 16:16:54.075141 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 12280
2026-08-06T16:16:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2895 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:16:54 dist-test-slave-t3q3 krb5kdc[13087](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:16:54 dist-test-slave-t3q3 krb5kdc[13087](info): set up 2 sockets
Aug 06 16:16:54 dist-test-slave-t3q3 krb5kdc[13087](info): commencing operation
krb5kdc: starting...
W20260806 16:16:55.385661 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:16:55 dist-test-slave-t3q3 krb5kdc[13087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:16:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:16:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:16:55.553072 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:33901
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:37763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:33901
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:16:55.703281 13108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:16:55.703672 13108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:16:55.703732 13108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:16:55.708751 13108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:16:55.708868 13108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:16:55.708889 13108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:16:55.708907 13108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:16:55.708925 13108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:16:55.715530 13108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:33901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:33901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:16:55.717284 13108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:16:55.718931 13108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:16:55.730521 13116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:16:55.730732 13108 server_base.cc:1034] running on GCE node
W20260806 16:16:55.730521 13114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:55.730521 13119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:16:55.731683 13108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:16:55.733383 13108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:16:55.734593 13108 hybrid_clock.cc:648] HybridClock initialized: now 1786033015734573 us; error 41 us; skew 500 ppm
Aug 06 16:16:55 dist-test-slave-t3q3 krb5kdc[13087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033015, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:16:55.738385 13108 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:16:55.739656 13108 webserver.cc:460] Webserver started at http://127.11.245.190:35355/ using document root <none> and password file <none>
I20260806 16:16:55.740608 13108 fs_manager.cc:359] Metadata directory not provided
I20260806 16:16:55.740696 13108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:16:55.741014 13108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:16:55.743782 13108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "ccc6269a630743008d0ec735de6f356b"
format_stamp: "Formatted at 2026-08-06 16:16:55 on dist-test-slave-t3q3"
server_key: "1fdf34e0a190c2993ff558d50cdedb50"
server_key_iv: "37676d40723c966b8f86389fce26a032"
server_key_version: "encryptionkey@0"
I20260806 16:16:55.744601 13108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "ccc6269a630743008d0ec735de6f356b"
format_stamp: "Formatted at 2026-08-06 16:16:55 on dist-test-slave-t3q3"
server_key: "1fdf34e0a190c2993ff558d50cdedb50"
server_key_iv: "37676d40723c966b8f86389fce26a032"
server_key_version: "encryptionkey@0"
I20260806 16:16:55.749938 13108 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 16:16:55.753369 13130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:55.755050 13108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:16:55.755231 13108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "ccc6269a630743008d0ec735de6f356b"
format_stamp: "Formatted at 2026-08-06 16:16:55 on dist-test-slave-t3q3"
server_key: "1fdf34e0a190c2993ff558d50cdedb50"
server_key_iv: "37676d40723c966b8f86389fce26a032"
server_key_version: "encryptionkey@0"
I20260806 16:16:55.755409 13108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:16:55.774669 13108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:16:55.779946 13108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:16:55.780264 13108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:16:55.789852 13108 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:33901
I20260806 16:16:55.789866 13206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:33901 every 8 connection(s)
I20260806 16:16:55.791208 13108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:16:55.794350 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 13108
I20260806 16:16:55.794500 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:16:55.794862 12246 external_mini_cluster.cc:1428] Setting key 35f51eca8bbae8b315df72ff26f4f17a
I20260806 16:16:55.795692 13208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:16:55.804782 13208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b: Bootstrap starting.
Aug 06 16:16:55 dist-test-slave-t3q3 krb5kdc[13087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:16:55.808542 13208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b: Neither blocks nor log segments found. Creating new log.
I20260806 16:16:55.809566 13208 log.cc:824] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b: Log is configured to *not* fsync() on all Append() calls
I20260806 16:16:55.812397 13208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b: No bootstrap required, opened a new log
I20260806 16:16:55.813154 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:55.796704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53858 (local address 127.11.245.190:33901)
0806 16:16:55.797254 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:55.797277 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:55.797334 (+ 57us) server_negotiation.cc:407] Connection header received
0806 16:16:55.798260 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:55.798283 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:55.798702 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:55.799080 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:55.800252 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:55.801404 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:55.802699 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:55.803297 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:55.803469 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:55.807051 (+ 3582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:55.807058 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:55.807078 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:55.807118 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:55.809498 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:55.810095 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:55.810101 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:55.810107 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:55.810258 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:55.810588 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:55.810591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:55.810592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:55.811124 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:55.811359 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:55.811705 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 16:16:55.812133 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":208,"threads_started":1}
I20260806 16:16:55.817612 13208 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccc6269a630743008d0ec735de6f356b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33901 } }
I20260806 16:16:55.818005 13208 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:16:55.818107 13208 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccc6269a630743008d0ec735de6f356b, State: Initialized, Role: FOLLOWER
I20260806 16:16:55.818785 13208 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccc6269a630743008d0ec735de6f356b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33901 } }
I20260806 16:16:55.819005 13208 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:16:55.819108 13208 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:16:55.819259 13208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:16:55.820698 13208 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccc6269a630743008d0ec735de6f356b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33901 } }
I20260806 16:16:55.821187 13208 leader_election.cc:302] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccc6269a630743008d0ec735de6f356b; no voters:
I20260806 16:16:55.821647 13208 leader_election.cc:288] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:16:55.821815 13215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:16:55.822132 13215 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [term 1 LEADER]: Becoming Leader. State: Replica: ccc6269a630743008d0ec735de6f356b, State: Running, Role: LEADER
I20260806 16:16:55.822566 13215 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccc6269a630743008d0ec735de6f356b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33901 } }
I20260806 16:16:55.823539 13208 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:16:55.825165 13216 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccc6269a630743008d0ec735de6f356b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccc6269a630743008d0ec735de6f356b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33901 } } }
I20260806 16:16:55.825203 13218 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccc6269a630743008d0ec735de6f356b. Latest consensus state: current_term: 1 leader_uuid: "ccc6269a630743008d0ec735de6f356b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccc6269a630743008d0ec735de6f356b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33901 } } }
I20260806 16:16:55.825336 13218 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [sys.catalog]: This master's current role is: LEADER
I20260806 16:16:55.825337 13216 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b [sys.catalog]: This master's current role is: LEADER
I20260806 16:16:55.825958 13227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:16:55.831045 13227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:16:55.840042 13227 catalog_manager.cc:1347] Generated new cluster ID: 2313c3a031f14a37aa4cdc66b11466cb
I20260806 16:16:55.840206 13227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:16:55.854162 13227 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:16:55.855590 13227 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:16:55.869239 13227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccc6269a630743008d0ec735de6f356b: Generated new TSK 0
I20260806 16:16:55.870369 13227 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:16:55.916733 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33901
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:37763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:16:56.063110 13241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:16:56.063460 13241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:16:56.063511 13241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:16:56.067832 13241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:16:56.067914 13241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:16:56.067978 13241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:16:56.073988 13241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33901
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:16:56.075578 13241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:16:56.077095 13241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:16:56.089442 13256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:56.089485 13254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:56.089496 13253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:16:56.089934 13241 server_base.cc:1034] running on GCE node
I20260806 16:16:56.090487 13241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:16:56.091359 13241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:16:56.092597 13241 hybrid_clock.cc:648] HybridClock initialized: now 1786033016092569 us; error 40 us; skew 500 ppm
Aug 06 16:16:56 dist-test-slave-t3q3 krb5kdc[13087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033016, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:16:56.096395 13241 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:16:56.097600 13241 webserver.cc:460] Webserver started at http://127.11.245.129:32943/ using document root <none> and password file <none>
I20260806 16:16:56.098371 13241 fs_manager.cc:359] Metadata directory not provided
I20260806 16:16:56.098476 13241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:16:56.098790 13241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:16:56.102872 13241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "6309c211071d4707be3da746b50959aa"
format_stamp: "Formatted at 2026-08-06 16:16:56 on dist-test-slave-t3q3"
server_key: "41f83069b566110bb1a2046ca260d77b"
server_key_iv: "d0f3c9476e39b110ed3c7c1cddaa7912"
server_key_version: "encryptionkey@0"
I20260806 16:16:56.103981 13241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "6309c211071d4707be3da746b50959aa"
format_stamp: "Formatted at 2026-08-06 16:16:56 on dist-test-slave-t3q3"
server_key: "41f83069b566110bb1a2046ca260d77b"
server_key_iv: "d0f3c9476e39b110ed3c7c1cddaa7912"
server_key_version: "encryptionkey@0"
I20260806 16:16:56.113090 13241 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.000s
I20260806 16:16:56.117302 13267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:56.119328 13241 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260806 16:16:56.119563 13241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "6309c211071d4707be3da746b50959aa"
format_stamp: "Formatted at 2026-08-06 16:16:56 on dist-test-slave-t3q3"
server_key: "41f83069b566110bb1a2046ca260d77b"
server_key_iv: "d0f3c9476e39b110ed3c7c1cddaa7912"
server_key_version: "encryptionkey@0"
I20260806 16:16:56.119755 13241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:16:56.162022 13241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:16:56.166901 13241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:16:56.167227 13241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:16:56.168313 13241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:16:56.169574 13241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:16:56.169660 13241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:56.169749 13241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:16:56.169795 13241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:56.181208 13241 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:43131
I20260806 16:16:56.181257 13436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:43131 every 8 connection(s)
I20260806 16:16:56.182523 13241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:16:56.189203 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 13241
I20260806 16:16:56.189349 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:16:56.189687 12246 external_mini_cluster.cc:1428] Setting key 6bd21a439f4c3b219b882e46884afd51
Aug 06 16:16:56 dist-test-slave-t3q3 krb5kdc[13087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033016, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:16:56.203653 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.185279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:53809 (local address 127.11.245.190:33901)
0806 16:16:56.185486 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:56.185493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:56.186682 (+ 1189us) server_negotiation.cc:407] Connection header received
0806 16:16:56.188296 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:56.188300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:56.188365 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:56.188478 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:56.190400 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.191280 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.193535 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.193819 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.193932 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.197733 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:56.197741 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:56.197749 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:56.197794 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:56.200104 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:56.201054 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:56.201058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:56.201059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:56.201160 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:56.201736 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:56.201740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:56.201741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:56.202021 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:56.202116 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:56.203088 (+ 972us) server_negotiation.cc:299] Negotiation successful
0806 16:16:56.203267 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":73}
I20260806 16:16:56.205377 13440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.185891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:33901 (local address 127.11.245.129:53809)
0806 16:16:56.186477 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:56.186547 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:56.187966 (+ 1419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:56.188698 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:56.188705 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:56.189327 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:16:56.190201 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:56.190232 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.191503 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.191509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:56.193320 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:56.193341 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.194037 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.194044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:56.194361 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.195302 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:16:56.195324 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:16:56.197454 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:16:56.200273 (+ 2819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:56.200280 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:56.200302 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:56.200877 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:56.201278 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:56.201285 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:56.201290 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:56.201565 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:56.202158 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:16:56.202172 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:16:56.202686 (+ 514us) client_negotiation.cc:769] Sending connection context
0806 16:16:56.203006 (+ 320us) client_negotiation.cc:240] Negotiation successful
0806 16:16:56.203660 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":142,"threads_started":1}
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:16:56.207258 13437 heartbeater.cc:347] Connected to a master server at 127.11.245.190:33901
I20260806 16:16:56.207628 13437 heartbeater.cc:464] Registering TS with master...
I20260806 16:16:56.208567 13437 heartbeater.cc:511] Master 127.11.245.190:33901 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
I20260806 16:16:56.211026 13158 ts_manager.cc:194] Registered new tserver with Master: 6309c211071d4707be3da746b50959aa (127.11.245.129:43131)
I20260806 16:16:56.213505 13158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:53809
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:16:56.232306 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33901
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:37763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:16:56.384855 13443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:16:56.385283 13443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:16:56.385371 13443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:16:56.390384 13443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:16:56.390543 13443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:16:56.390661 13443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:16:56.397734 13443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33901
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:16:56.399398 13443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:16:56.400846 13443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:16:56.413619 13451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:56.413622 13455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:56.413681 13452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:16:56.414196 13443 server_base.cc:1034] running on GCE node
I20260806 16:16:56.414724 13443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:16:56.415570 13443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:16:56.416756 13443 hybrid_clock.cc:648] HybridClock initialized: now 1786033016416740 us; error 33 us; skew 500 ppm
Aug 06 16:16:56 dist-test-slave-t3q3 krb5kdc[13087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033016, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:16:56.420291 13443 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:16:56.421443 13443 webserver.cc:460] Webserver started at http://127.11.245.130:38483/ using document root <none> and password file <none>
I20260806 16:16:56.422195 13443 fs_manager.cc:359] Metadata directory not provided
I20260806 16:16:56.422264 13443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:16:56.422590 13443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:16:56.425704 13443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "472e98de2a12418da539442bf9d2563f"
format_stamp: "Formatted at 2026-08-06 16:16:56 on dist-test-slave-t3q3"
server_key: "3031ad6c46366f6a99fb9c796465b8d2"
server_key_iv: "1072e54a5ac164c10238db035f0d1bbb"
server_key_version: "encryptionkey@0"
I20260806 16:16:56.426600 13443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "472e98de2a12418da539442bf9d2563f"
format_stamp: "Formatted at 2026-08-06 16:16:56 on dist-test-slave-t3q3"
server_key: "3031ad6c46366f6a99fb9c796465b8d2"
server_key_iv: "1072e54a5ac164c10238db035f0d1bbb"
server_key_version: "encryptionkey@0"
I20260806 16:16:56.431756 13443 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 16:16:56.435437 13464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:56.437378 13443 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:16:56.437585 13443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "472e98de2a12418da539442bf9d2563f"
format_stamp: "Formatted at 2026-08-06 16:16:56 on dist-test-slave-t3q3"
server_key: "3031ad6c46366f6a99fb9c796465b8d2"
server_key_iv: "1072e54a5ac164c10238db035f0d1bbb"
server_key_version: "encryptionkey@0"
I20260806 16:16:56.437770 13443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:16:56.458374 13443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:16:56.463217 13443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:16:56.463503 13443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:16:56.464389 13443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:16:56.465591 13443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:16:56.465685 13443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:56.465776 13443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:16:56.465827 13443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:56.477562 13443 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:42467
I20260806 16:16:56.477591 13634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:42467 every 8 connection(s)
I20260806 16:16:56.478809 13443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:16:56.484216 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 13443
I20260806 16:16:56.484364 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:16:56.484745 12246 external_mini_cluster.cc:1428] Setting key 1a1b87466c1c4540b3d1b6534e4f92f8
Aug 06 16:16:56 dist-test-slave-t3q3 krb5kdc[13087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033016, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:16:56.496531 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.481295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:37971 (local address 127.11.245.190:33901)
0806 16:16:56.481493 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:56.481500 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:56.482728 (+ 1228us) server_negotiation.cc:407] Connection header received
0806 16:16:56.484578 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:56.484583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:56.484649 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:56.484725 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:56.486507 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.487080 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.488415 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.488669 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.488791 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.491945 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:56.491949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:56.491951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:56.491980 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:56.493718 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:56.494458 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:56.494461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:56.494468 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:56.494573 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:56.495071 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:56.495074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:56.495076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:56.495295 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:56.495383 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:56.496214 (+ 831us) server_negotiation.cc:299] Negotiation successful
0806 16:16:56.496350 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 16:16:56.498188 13638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.481836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:33901 (local address 127.11.245.130:37971)
0806 16:16:56.482512 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:56.482567 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:56.484227 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:56.485008 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:56.485018 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:56.485617 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:16:56.486349 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:56.486373 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.487218 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.487221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:56.488286 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:56.488297 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.488850 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.488855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:56.489057 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.489856 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:16:56.489876 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:16:56.491735 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:16:56.493855 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:56.493865 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:56.493887 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:56.494313 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:56.494693 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:56.494699 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:56.494703 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:56.494932 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:56.495419 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:16:56.495427 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:16:56.495870 (+ 443us) client_negotiation.cc:769] Sending connection context
0806 16:16:56.496166 (+ 296us) client_negotiation.cc:240] Negotiation successful
0806 16:16:56.496655 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":156,"threads_started":1}
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
I20260806 16:16:56.499724 13635 heartbeater.cc:347] Connected to a master server at 127.11.245.190:33901
I20260806 16:16:56.500237 13635 heartbeater.cc:464] Registering TS with master...
I20260806 16:16:56.501032 13635 heartbeater.cc:511] Master 127.11.245.190:33901 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
I20260806 16:16:56.502667 13158 ts_manager.cc:194] Registered new tserver with Master: 472e98de2a12418da539442bf9d2563f (127.11.245.130:42467)
I20260806 16:16:56.503742 13158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:37971
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:16:56.523885 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33901
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:37763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:16:56.665844 13641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:16:56.666242 13641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:16:56.666299 13641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:16:56.670975 13641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:16:56.671069 13641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:16:56.671140 13641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:16:56.677517 13641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33901
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:16:56.679472 13641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:16:56.681238 13641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:16:56.694877 13650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:56.694833 13646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:56.694808 13647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:16:56.695379 13641 server_base.cc:1034] running on GCE node
I20260806 16:16:56.695959 13641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:16:56.697223 13641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:16:56.698474 13641 hybrid_clock.cc:648] HybridClock initialized: now 1786033016698431 us; error 55 us; skew 500 ppm
Aug 06 16:16:56 dist-test-slave-t3q3 krb5kdc[13087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033016, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:16:56.702186 13641 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:16:56.703526 13641 webserver.cc:460] Webserver started at http://127.11.245.131:36887/ using document root <none> and password file <none>
I20260806 16:16:56.704438 13641 fs_manager.cc:359] Metadata directory not provided
I20260806 16:16:56.704543 13641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:16:56.704854 13641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:16:56.708317 13641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "39824d70f4ef4cad89214e6f7e7b0276"
format_stamp: "Formatted at 2026-08-06 16:16:56 on dist-test-slave-t3q3"
server_key: "ba8425eb1f400d3b17afa5f596451be8"
server_key_iv: "4fe9d79d0dd0385c81cab3d74a8ee2a1"
server_key_version: "encryptionkey@0"
I20260806 16:16:56.709182 13641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "39824d70f4ef4cad89214e6f7e7b0276"
format_stamp: "Formatted at 2026-08-06 16:16:56 on dist-test-slave-t3q3"
server_key: "ba8425eb1f400d3b17afa5f596451be8"
server_key_iv: "4fe9d79d0dd0385c81cab3d74a8ee2a1"
server_key_version: "encryptionkey@0"
I20260806 16:16:56.714694 13641 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:16:56.718139 13658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:56.719822 13641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:16:56.720008 13641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "39824d70f4ef4cad89214e6f7e7b0276"
format_stamp: "Formatted at 2026-08-06 16:16:56 on dist-test-slave-t3q3"
server_key: "ba8425eb1f400d3b17afa5f596451be8"
server_key_iv: "4fe9d79d0dd0385c81cab3d74a8ee2a1"
server_key_version: "encryptionkey@0"
I20260806 16:16:56.720229 13641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:16:56.734995 13641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:16:56.741736 13641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:16:56.742067 13641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:16:56.743033 13641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:16:56.744323 13641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:16:56.744407 13641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:56.744493 13641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:16:56.744539 13641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:56.758111 13641 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:38609
I20260806 16:16:56.758172 13836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:38609 every 8 connection(s)
I20260806 16:16:56.759613 13641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:16:56.765327 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 13641
I20260806 16:16:56.765507 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:16:56.765899 12246 external_mini_cluster.cc:1428] Setting key 90ae0fc1356a27113d858fdfbc6f31c2
Aug 06 16:16:56 dist-test-slave-t3q3 krb5kdc[13087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033016, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:16:56.779538 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.762436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:51413 (local address 127.11.245.190:33901)
0806 16:16:56.762648 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:56.762653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:56.763879 (+ 1226us) server_negotiation.cc:407] Connection header received
0806 16:16:56.766097 (+ 2218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:56.766105 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:56.766185 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:56.766277 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:56.768447 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.769216 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.770774 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.771020 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.771139 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.774598 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:56.774603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:56.774605 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:56.774640 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:56.776870 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:56.777572 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:56.777576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:56.777578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:56.777677 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:56.778101 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:56.778105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:56.778107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:56.778355 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:56.778476 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:56.779157 (+ 681us) server_negotiation.cc:299] Negotiation successful
0806 16:16:56.779323 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 16:16:56.781055 13840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.762927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:33901 (local address 127.11.245.131:51413)
0806 16:16:56.763654 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:56.763715 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:56.765663 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:56.766597 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:56.766611 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:56.767317 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:16:56.768254 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:56.768279 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.769431 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.769435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:56.770616 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:56.770629 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.771257 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.771261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:56.771497 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.772359 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:16:56.772381 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:16:56.774327 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:16:56.777061 (+ 2734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:56.777070 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:56.777091 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:56.777434 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:56.777784 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:56.777787 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:56.777789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:56.777984 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:56.778500 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:16:56.778516 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:16:56.778863 (+ 347us) client_negotiation.cc:769] Sending connection context
0806 16:16:56.779104 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 16:16:56.779556 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":173,"threads_started":1}
I20260806 16:16:56.782464 13837 heartbeater.cc:347] Connected to a master server at 127.11.245.190:33901
I20260806 16:16:56.782815 13837 heartbeater.cc:464] Registering TS with master...
I20260806 16:16:56.783555 13837 heartbeater.cc:511] Master 127.11.245.190:33901 requested a full tablet report, sending...
I20260806 16:16:56.785143 13158 ts_manager.cc:194] Registered new tserver with Master: 39824d70f4ef4cad89214e6f7e7b0276 (127.11.245.131:38609)
I20260806 16:16:56.786082 13158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:51413
I20260806 16:16:56.794517 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:16:56.815047 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.800460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53870 (local address 127.11.245.190:33901)
0806 16:16:56.800715 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:56.800721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:56.801438 (+ 717us) server_negotiation.cc:407] Connection header received
0806 16:16:56.801686 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:56.801690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:56.801771 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:56.801870 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:56.803065 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.804119 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.806121 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.806536 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.806683 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.808896 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:56.808903 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:56.808908 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:56.808956 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:56.812543 (+ 3587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:56.813096 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:56.813102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:56.813108 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:56.813216 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:56.813528 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:56.813532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:56.813556 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:56.813970 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:56.814172 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:56.814636 (+ 464us) server_negotiation.cc:299] Negotiation successful
0806 16:16:56.814811 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 16:16:56.823634 13158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:16:56.828989 13158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:16:56.851491 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.843726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34410 (local address 127.11.245.130:42467)
0806 16:16:56.844454 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:56.844470 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:56.844507 (+ 37us) server_negotiation.cc:407] Connection header received
0806 16:16:56.844608 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:56.844614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:56.844887 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:56.845090 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:16:56.845835 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.847219 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.849329 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.850277 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.850412 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.850603 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:56.851111 (+ 508us) server_negotiation.cc:299] Negotiation successful
0806 16:16:56.851255 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":176,"threads_started":1}
I20260806 16:16:56.852481 13861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.843561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:42467 (local address 127.0.0.1:34410)
0806 16:16:56.844121 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:56.844194 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:56.844381 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:56.845401 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:56.845409 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:56.845454 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:16:56.845658 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:56.845668 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.847441 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.847444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:56.849171 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:56.849180 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.850712 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.850716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:56.850847 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.850893 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 16:16:56.852134 (+ 1241us) client_negotiation.cc:240] Negotiation successful
0806 16:16:56.852325 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":136,"threads_started":1}
I20260806 16:16:56.852600 13865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.845335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54004 (local address 127.11.245.131:38609)
0806 16:16:56.845769 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:56.845786 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:56.846106 (+ 320us) server_negotiation.cc:407] Connection header received
0806 16:16:56.846214 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:56.846221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:56.846454 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:56.846622 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:16:56.847466 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.848872 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.850427 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.851596 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.851752 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.851901 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:56.852302 (+ 401us) server_negotiation.cc:299] Negotiation successful
0806 16:16:56.852422 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":188,"threads_started":1}
I20260806 16:16:56.853034 13864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.845091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:38609 (local address 127.0.0.1:54004)
0806 16:16:56.845851 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:56.845868 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:56.845997 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:56.846801 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:56.846804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:56.846825 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:16:56.847305 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:56.847320 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.849059 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.849062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:56.850268 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:56.850278 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.851974 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.851978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:56.852107 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.852134 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 16:16:56.852207 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 16:16:56.852928 (+ 721us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":625,"thread_start_us":119,"threads_started":1}
I20260806 16:16:56.855032 13866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.845404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:43131 (local address 127.0.0.1:35056)
0806 16:16:56.846650 (+ 1246us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:56.846667 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:56.846777 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:56.848433 (+ 1656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:56.848438 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:56.848466 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:16:56.848623 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:56.848634 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.850082 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.850086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:56.852528 (+ 2442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:56.852543 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.854223 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.854228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:56.854362 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.854399 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 16:16:56.854488 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 16:16:56.854834 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1171,"thread_start_us":133,"threads_started":1}
I20260806 16:16:56.855793 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.845951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35056 (local address 127.11.245.129:43131)
0806 16:16:56.846289 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:56.846296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:56.846900 (+ 604us) server_negotiation.cc:407] Connection header received
0806 16:16:56.847029 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:56.847037 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:56.848011 (+ 974us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:56.848213 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:16:56.848789 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.849930 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.852713 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.853789 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.853955 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.854136 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:56.855389 (+ 1253us) server_negotiation.cc:299] Negotiation successful
0806 16:16:56.855559 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":136,"threads_started":1}
I20260806 16:16:56.857928 13747 tablet_service.cc:1467] Processing CreateTablet for tablet f9e498e398bd477f9212a504179fb8c6 (DEFAULT_TABLE table=test-table [id=15adbd7dbb784ae0b803fc2cbed44774]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:16:56.858553 13539 tablet_service.cc:1467] Processing CreateTablet for tablet f9e498e398bd477f9212a504179fb8c6 (DEFAULT_TABLE table=test-table [id=15adbd7dbb784ae0b803fc2cbed44774]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:16:56.859640 13747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9e498e398bd477f9212a504179fb8c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:16:56.859382 13329 tablet_service.cc:1467] Processing CreateTablet for tablet f9e498e398bd477f9212a504179fb8c6 (DEFAULT_TABLE table=test-table [id=15adbd7dbb784ae0b803fc2cbed44774]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:16:56.860838 13329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9e498e398bd477f9212a504179fb8c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:16:56.861267 13539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9e498e398bd477f9212a504179fb8c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:16:56.869859 13868 tablet_bootstrap.cc:492] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276: Bootstrap starting.
I20260806 16:16:56.871800 13870 tablet_bootstrap.cc:492] T f9e498e398bd477f9212a504179fb8c6 P 472e98de2a12418da539442bf9d2563f: Bootstrap starting.
I20260806 16:16:56.873217 13868 tablet_bootstrap.cc:654] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276: Neither blocks nor log segments found. Creating new log.
I20260806 16:16:56.875451 13868 log.cc:824] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276: Log is configured to *not* fsync() on all Append() calls
I20260806 16:16:56.875545 13870 tablet_bootstrap.cc:654] T f9e498e398bd477f9212a504179fb8c6 P 472e98de2a12418da539442bf9d2563f: Neither blocks nor log segments found. Creating new log.
I20260806 16:16:56.876672 13869 tablet_bootstrap.cc:492] T f9e498e398bd477f9212a504179fb8c6 P 6309c211071d4707be3da746b50959aa: Bootstrap starting.
I20260806 16:16:56.876816 13870 log.cc:824] T f9e498e398bd477f9212a504179fb8c6 P 472e98de2a12418da539442bf9d2563f: Log is configured to *not* fsync() on all Append() calls
I20260806 16:16:56.878952 13868 tablet_bootstrap.cc:492] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276: No bootstrap required, opened a new log
I20260806 16:16:56.879261 13868 ts_tablet_manager.cc:1397] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260806 16:16:56.879424 13870 tablet_bootstrap.cc:492] T f9e498e398bd477f9212a504179fb8c6 P 472e98de2a12418da539442bf9d2563f: No bootstrap required, opened a new log
I20260806 16:16:56.879642 13870 ts_tablet_manager.cc:1397] T f9e498e398bd477f9212a504179fb8c6 P 472e98de2a12418da539442bf9d2563f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 16:16:56.880613 13869 tablet_bootstrap.cc:654] T f9e498e398bd477f9212a504179fb8c6 P 6309c211071d4707be3da746b50959aa: Neither blocks nor log segments found. Creating new log.
I20260806 16:16:56.881937 13869 log.cc:824] T f9e498e398bd477f9212a504179fb8c6 P 6309c211071d4707be3da746b50959aa: Log is configured to *not* fsync() on all Append() calls
I20260806 16:16:56.883602 13868 raft_consensus.cc:348] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472e98de2a12418da539442bf9d2563f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42467 } } peers { permanent_uuid: "39824d70f4ef4cad89214e6f7e7b0276" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 38609 } } peers { permanent_uuid: "6309c211071d4707be3da746b50959aa" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43131 } }
I20260806 16:16:56.883939 13868 raft_consensus.cc:374] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:16:56.883980 13868 raft_consensus.cc:729] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39824d70f4ef4cad89214e6f7e7b0276, State: Initialized, Role: FOLLOWER
I20260806 16:16:56.884538 13870 raft_consensus.cc:348] T f9e498e398bd477f9212a504179fb8c6 P 472e98de2a12418da539442bf9d2563f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472e98de2a12418da539442bf9d2563f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42467 } } peers { permanent_uuid: "39824d70f4ef4cad89214e6f7e7b0276" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 38609 } } peers { permanent_uuid: "6309c211071d4707be3da746b50959aa" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43131 } }
I20260806 16:16:56.884721 13868 consensus_queue.cc:260] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472e98de2a12418da539442bf9d2563f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42467 } } peers { permanent_uuid: "39824d70f4ef4cad89214e6f7e7b0276" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 38609 } } peers { permanent_uuid: "6309c211071d4707be3da746b50959aa" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43131 } }
I20260806 16:16:56.884860 13870 raft_consensus.cc:374] T f9e498e398bd477f9212a504179fb8c6 P 472e98de2a12418da539442bf9d2563f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:16:56.884931 13870 raft_consensus.cc:729] T f9e498e398bd477f9212a504179fb8c6 P 472e98de2a12418da539442bf9d2563f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 472e98de2a12418da539442bf9d2563f, State: Initialized, Role: FOLLOWER
I20260806 16:16:56.885703 13837 heartbeater.cc:503] Master 127.11.245.190:33901 was elected leader, sending a full tablet report...
I20260806 16:16:56.886142 13868 ts_tablet_manager.cc:1428] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 16:16:56.886931 13869 tablet_bootstrap.cc:492] T f9e498e398bd477f9212a504179fb8c6 P 6309c211071d4707be3da746b50959aa: No bootstrap required, opened a new log
I20260806 16:16:56.887219 13869 ts_tablet_manager.cc:1397] T f9e498e398bd477f9212a504179fb8c6 P 6309c211071d4707be3da746b50959aa: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260806 16:16:56.887195 13870 consensus_queue.cc:260] T f9e498e398bd477f9212a504179fb8c6 P 472e98de2a12418da539442bf9d2563f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472e98de2a12418da539442bf9d2563f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42467 } } peers { permanent_uuid: "39824d70f4ef4cad89214e6f7e7b0276" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 38609 } } peers { permanent_uuid: "6309c211071d4707be3da746b50959aa" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43131 } }
I20260806 16:16:56.888162 13635 heartbeater.cc:503] Master 127.11.245.190:33901 was elected leader, sending a full tablet report...
I20260806 16:16:56.889858 13870 ts_tablet_manager.cc:1428] T f9e498e398bd477f9212a504179fb8c6 P 472e98de2a12418da539442bf9d2563f: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260806 16:16:56.892786 13869 raft_consensus.cc:348] T f9e498e398bd477f9212a504179fb8c6 P 6309c211071d4707be3da746b50959aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472e98de2a12418da539442bf9d2563f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42467 } } peers { permanent_uuid: "39824d70f4ef4cad89214e6f7e7b0276" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 38609 } } peers { permanent_uuid: "6309c211071d4707be3da746b50959aa" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43131 } }
I20260806 16:16:56.893215 13869 raft_consensus.cc:374] T f9e498e398bd477f9212a504179fb8c6 P 6309c211071d4707be3da746b50959aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:16:56.893291 13869 raft_consensus.cc:729] T f9e498e398bd477f9212a504179fb8c6 P 6309c211071d4707be3da746b50959aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6309c211071d4707be3da746b50959aa, State: Initialized, Role: FOLLOWER
I20260806 16:16:56.893896 13869 consensus_queue.cc:260] T f9e498e398bd477f9212a504179fb8c6 P 6309c211071d4707be3da746b50959aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472e98de2a12418da539442bf9d2563f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42467 } } peers { permanent_uuid: "39824d70f4ef4cad89214e6f7e7b0276" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 38609 } } peers { permanent_uuid: "6309c211071d4707be3da746b50959aa" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43131 } }
I20260806 16:16:56.894747 13437 heartbeater.cc:503] Master 127.11.245.190:33901 was elected leader, sending a full tablet report...
I20260806 16:16:56.895215 13869 ts_tablet_manager.cc:1428] T f9e498e398bd477f9212a504179fb8c6 P 6309c211071d4707be3da746b50959aa: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
W20260806 16:16:56.934460 13438 tablet.cc:2369] T f9e498e398bd477f9212a504179fb8c6 P 6309c211071d4707be3da746b50959aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:16:56.952337 13878 raft_consensus.cc:482] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:16:56.952643 13878 raft_consensus.cc:504] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472e98de2a12418da539442bf9d2563f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42467 } } peers { permanent_uuid: "39824d70f4ef4cad89214e6f7e7b0276" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 38609 } } peers { permanent_uuid: "6309c211071d4707be3da746b50959aa" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43131 } }
I20260806 16:16:56.954355 13878 leader_election.cc:288] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 472e98de2a12418da539442bf9d2563f (127.11.245.130:42467), 6309c211071d4707be3da746b50959aa (127.11.245.129:43131)
I20260806 16:16:56.960325 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.954855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:53371 (local address 127.11.245.130:42467)
0806 16:16:56.955078 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:56.955085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:56.955443 (+ 358us) server_negotiation.cc:407] Connection header received
0806 16:16:56.955530 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:56.955536 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:56.955607 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:56.955706 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:16:56.955986 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.956966 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.959069 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.959564 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.959685 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.959790 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:56.960014 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 16:16:56.960162 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260806 16:16:56.960350 13840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.954682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:43131 (local address 127.11.245.131:33887)
0806 16:16:56.954960 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:56.954980 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:56.955109 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:56.955592 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:56.955595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:56.955612 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:16:56.955810 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:56.955821 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.957119 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.957123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:56.958856 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:56.958862 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.959800 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.959807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:56.959992 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.960016 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 16:16:56.960162 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 16:16:56.960240 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":106}
I20260806 16:16:56.960362 13884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.954694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:42467 (local address 127.11.245.131:53371)
0806 16:16:56.955314 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:56.955326 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:56.955431 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:56.955717 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:56.955720 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:56.955734 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:16:56.955850 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:56.955857 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.957119 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.957123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:56.958846 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:56.958862 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.959691 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.959698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:56.959850 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.959880 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 16:16:56.959964 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 16:16:56.960043 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":241,"thread_start_us":196,"threads_started":1}
I20260806 16:16:56.960502 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:56.954860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:33887 (local address 127.11.245.129:43131)
0806 16:16:56.955034 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:56.955040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:56.955059 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:16:56.955342 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:56.955346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:56.955456 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:56.955570 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:16:56.955991 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.956952 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.959061 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:56.959655 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:56.959791 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:56.959870 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:56.960236 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 16:16:56.960345 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 16:16:56.961345 13365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e498e398bd477f9212a504179fb8c6" candidate_uuid: "39824d70f4ef4cad89214e6f7e7b0276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6309c211071d4707be3da746b50959aa" is_pre_election: true
I20260806 16:16:56.961340 13572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e498e398bd477f9212a504179fb8c6" candidate_uuid: "39824d70f4ef4cad89214e6f7e7b0276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "472e98de2a12418da539442bf9d2563f" is_pre_election: true
I20260806 16:16:56.961768 13365 raft_consensus.cc:2454] T f9e498e398bd477f9212a504179fb8c6 P 6309c211071d4707be3da746b50959aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39824d70f4ef4cad89214e6f7e7b0276 in term 0.
I20260806 16:16:56.961768 13572 raft_consensus.cc:2454] T f9e498e398bd477f9212a504179fb8c6 P 472e98de2a12418da539442bf9d2563f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39824d70f4ef4cad89214e6f7e7b0276 in term 0.
I20260806 16:16:56.962436 13660 leader_election.cc:302] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 39824d70f4ef4cad89214e6f7e7b0276, 472e98de2a12418da539442bf9d2563f, 6309c211071d4707be3da746b50959aa; no voters:
I20260806 16:16:56.962809 13878 raft_consensus.cc:2781] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:16:56.962942 13878 raft_consensus.cc:482] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:16:56.963027 13878 raft_consensus.cc:3037] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:16:56.964515 13878 raft_consensus.cc:504] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472e98de2a12418da539442bf9d2563f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42467 } } peers { permanent_uuid: "39824d70f4ef4cad89214e6f7e7b0276" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 38609 } } peers { permanent_uuid: "6309c211071d4707be3da746b50959aa" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43131 } }
I20260806 16:16:56.965058 13878 leader_election.cc:288] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [CANDIDATE]: Term 1 election: Requested vote from peers 472e98de2a12418da539442bf9d2563f (127.11.245.130:42467), 6309c211071d4707be3da746b50959aa (127.11.245.129:43131)
I20260806 16:16:56.965685 13572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e498e398bd477f9212a504179fb8c6" candidate_uuid: "39824d70f4ef4cad89214e6f7e7b0276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "472e98de2a12418da539442bf9d2563f"
I20260806 16:16:56.965720 13365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e498e398bd477f9212a504179fb8c6" candidate_uuid: "39824d70f4ef4cad89214e6f7e7b0276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6309c211071d4707be3da746b50959aa"
I20260806 16:16:56.965902 13572 raft_consensus.cc:3037] T f9e498e398bd477f9212a504179fb8c6 P 472e98de2a12418da539442bf9d2563f [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:16:56.965902 13365 raft_consensus.cc:3037] T f9e498e398bd477f9212a504179fb8c6 P 6309c211071d4707be3da746b50959aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:16:56.967790 13365 raft_consensus.cc:2454] T f9e498e398bd477f9212a504179fb8c6 P 6309c211071d4707be3da746b50959aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39824d70f4ef4cad89214e6f7e7b0276 in term 1.
I20260806 16:16:56.967796 13572 raft_consensus.cc:2454] T f9e498e398bd477f9212a504179fb8c6 P 472e98de2a12418da539442bf9d2563f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39824d70f4ef4cad89214e6f7e7b0276 in term 1.
I20260806 16:16:56.968351 13665 leader_election.cc:302] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39824d70f4ef4cad89214e6f7e7b0276, 472e98de2a12418da539442bf9d2563f; no voters:
I20260806 16:16:56.968587 13878 raft_consensus.cc:2781] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:16:56.968986 13878 raft_consensus.cc:686] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [term 1 LEADER]: Becoming Leader. State: Replica: 39824d70f4ef4cad89214e6f7e7b0276, State: Running, Role: LEADER
I20260806 16:16:56.969463 13878 consensus_queue.cc:237] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472e98de2a12418da539442bf9d2563f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42467 } } peers { permanent_uuid: "39824d70f4ef4cad89214e6f7e7b0276" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 38609 } } peers { permanent_uuid: "6309c211071d4707be3da746b50959aa" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43131 } }
I20260806 16:16:56.974237 13158 catalog_manager.cc:5495] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39824d70f4ef4cad89214e6f7e7b0276 (127.11.245.131). New cstate: current_term: 1 leader_uuid: "39824d70f4ef4cad89214e6f7e7b0276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472e98de2a12418da539442bf9d2563f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39824d70f4ef4cad89214e6f7e7b0276" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 38609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6309c211071d4707be3da746b50959aa" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43131 } health_report { overall_health: UNKNOWN } } }
W20260806 16:16:56.980372 13637 tablet.cc:2369] T f9e498e398bd477f9212a504179fb8c6 P 472e98de2a12418da539442bf9d2563f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 16:16:57 dist-test-slave-t3q3 krb5kdc[13087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.129@KRBTEST.COM
W20260806 16:16:57.011179 13838 tablet.cc:2369] T f9e498e398bd477f9212a504179fb8c6 P 39824d70f4ef4cad89214e6f7e7b0276: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:16:57.014106 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:57.002122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35062 (local address 127.11.245.129:43131)
0806 16:16:57.002366 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:57.002372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:57.002523 (+ 151us) server_negotiation.cc:407] Connection header received
0806 16:16:57.002675 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:57.002678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:57.002735 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:57.002820 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:57.003841 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:57.004567 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:57.005544 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:57.005782 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:57.005886 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:57.009218 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:57.009242 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:57.009270 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:57.009318 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:57.011676 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:57.012362 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:57.012397 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:57.012398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:57.012519 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:57.012828 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:57.012831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:57.012832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:57.013206 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:57.013436 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:57.013680 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 16:16:57.013813 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
Aug 06 16:16:57 dist-test-slave-t3q3 krb5kdc[13087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033017, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 16:16:57 dist-test-slave-t3q3 krb5kdc[13087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033017, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.245.129@KRBTEST.COM
I20260806 16:16:57.051759 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:57.040786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35070 (local address 127.11.245.129:43131)
0806 16:16:57.041105 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:57.041113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:57.041137 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:16:57.041266 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:57.041270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:57.041328 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:57.041408 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:57.042241 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:57.042875 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:57.043886 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:57.044222 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:57.044332 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:57.047694 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:57.047698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:57.047701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:57.047740 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:57.049540 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:57.050178 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:57.050182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:57.050187 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:57.050290 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:57.050599 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:57.050603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:57.050605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:57.050836 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:57.051023 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:57.051355 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 16:16:57.051523 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
Aug 06 16:16:57 dist-test-slave-t3q3 krb5kdc[13087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 16:16:57 dist-test-slave-t3q3 krb5kdc[13087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.129@KRBTEST.COM
I20260806 16:16:57.089401 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:57.078175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35080 (local address 127.11.245.129:43131)
0806 16:16:57.078455 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:57.078461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:57.078601 (+ 140us) server_negotiation.cc:407] Connection header received
0806 16:16:57.078810 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:57.078818 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:57.078890 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:57.078992 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:57.079803 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:57.080448 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:57.081373 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:57.081606 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:57.081718 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:57.084729 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:57.084733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:57.084735 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:57.084767 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:57.086714 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:57.087454 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:57.087459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:57.087461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:57.087572 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:57.088115 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:57.088122 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:57.088124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:57.088513 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:57.088699 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:57.089010 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 16:16:57.089172 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
W20260806 16:16:57.091566 13301 tablet_service.cc:2908] Rejecting scan request for tablet f9e498e398bd477f9212a504179fb8c6: Uninitialized: clean time has not yet been initialized
I20260806 16:16:57.094925 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 13241
I20260806 16:16:57.114560 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 13443
I20260806 16:16:57.138957 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 13641
I20260806 16:16:57.156242 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 13108
2026-08-06T16:16:57Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (3078 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:16:57 dist-test-slave-t3q3 krb5kdc[13907](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:16:57 dist-test-slave-t3q3 krb5kdc[13907](info): set up 2 sockets
Aug 06 16:16:57 dist-test-slave-t3q3 krb5kdc[13907](info): commencing operation
krb5kdc: starting...
W20260806 16:16:58.441309 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:16:58 dist-test-slave-t3q3 krb5kdc[13907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:16:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:16:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:16:58.600351 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:43743
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:43743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:16:58.759285 13930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:16:58.759907 13930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:16:58.759989 13930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:16:58.766791 13930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:16:58.766904 13930 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 16:16:58.766932 13930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:16:58.766960 13930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:16:58.767004 13930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:16:58.767033 13930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:16:58.779093 13930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:43743
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:43743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:16:58.781280 13930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:16:58.782750 13930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:16:58.794260 13944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:58.794298 13942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:58.794260 13946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:16:58.794806 13930 server_base.cc:1034] running on GCE node
I20260806 16:16:58.795562 13930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:16:58.797132 13930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:16:58.798350 13930 hybrid_clock.cc:648] HybridClock initialized: now 1786033018798318 us; error 44 us; skew 500 ppm
Aug 06 16:16:58 dist-test-slave-t3q3 krb5kdc[13907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033018, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:16:58.801931 13930 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:16:58.803213 13930 webserver.cc:460] Webserver started at http://127.11.245.190:36675/ using document root <none> and password file <none>
I20260806 16:16:58.803987 13930 fs_manager.cc:359] Metadata directory not provided
I20260806 16:16:58.804119 13930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:16:58.804411 13930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:16:58.807191 13930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "3641097f8d954eaf9fbf075936b18b28"
format_stamp: "Formatted at 2026-08-06 16:16:58 on dist-test-slave-t3q3"
server_key: "8cf90d6fb795a67046e2d1aab66404d2"
server_key_iv: "ba283d5482c8ced886780deb6d73b2ae"
server_key_version: "encryptionkey@0"
I20260806 16:16:58.807832 13930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "3641097f8d954eaf9fbf075936b18b28"
format_stamp: "Formatted at 2026-08-06 16:16:58 on dist-test-slave-t3q3"
server_key: "8cf90d6fb795a67046e2d1aab66404d2"
server_key_iv: "ba283d5482c8ced886780deb6d73b2ae"
server_key_version: "encryptionkey@0"
I20260806 16:16:58.812692 13930 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:16:58.815838 13955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:58.817600 13930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:16:58.817768 13930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "3641097f8d954eaf9fbf075936b18b28"
format_stamp: "Formatted at 2026-08-06 16:16:58 on dist-test-slave-t3q3"
server_key: "8cf90d6fb795a67046e2d1aab66404d2"
server_key_iv: "ba283d5482c8ced886780deb6d73b2ae"
server_key_version: "encryptionkey@0"
I20260806 16:16:58.817916 13930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:16:58.840627 13930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:16:58.844913 13930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:16:58.845180 13930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:16:58.855953 13930 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:43743
I20260806 16:16:58.855963 14044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:43743 every 8 connection(s)
I20260806 16:16:58.857319 13930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:16:58.861223 14045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:16:58.861881 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 13930
I20260806 16:16:58.862025 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:16:58.862382 12246 external_mini_cluster.cc:1428] Setting key a6d327459dbf8c5a6cc8fb809c4e2ef8
I20260806 16:16:58.870682 14045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28: Bootstrap starting.
Aug 06 16:16:58 dist-test-slave-t3q3 krb5kdc[13907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:16:58.874814 14045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28: Neither blocks nor log segments found. Creating new log.
I20260806 16:16:58.876626 14045 log.cc:824] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28: Log is configured to *not* fsync() on all Append() calls
I20260806 16:16:58.879993 14045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28: No bootstrap required, opened a new log
I20260806 16:16:58.881330 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:58.864279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39094 (local address 127.11.245.190:43743)
0806 16:16:58.864839 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:58.864858 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:58.864903 (+ 45us) server_negotiation.cc:407] Connection header received
0806 16:16:58.865707 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:58.865729 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:58.866170 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:58.866524 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:58.867671 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:58.868891 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:58.870176 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:58.870798 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:58.871061 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:58.874577 (+ 3516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:58.874587 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:58.874614 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:58.874662 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:58.877187 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:58.877908 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:58.877916 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:58.877924 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:58.878089 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:58.878440 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:58.878442 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:58.878444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:58.879066 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:58.879269 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:58.879644 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 16:16:58.880013 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":183,"threads_started":1}
I20260806 16:16:58.884569 14045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3641097f8d954eaf9fbf075936b18b28" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43743 } }
I20260806 16:16:58.884848 14045 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:16:58.884884 14045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3641097f8d954eaf9fbf075936b18b28, State: Initialized, Role: FOLLOWER
I20260806 16:16:58.885398 14045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3641097f8d954eaf9fbf075936b18b28" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43743 } }
I20260806 16:16:58.885551 14045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:16:58.885602 14045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:16:58.885684 14045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:16:58.886803 14045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3641097f8d954eaf9fbf075936b18b28" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43743 } }
I20260806 16:16:58.887167 14045 leader_election.cc:302] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3641097f8d954eaf9fbf075936b18b28; no voters:
I20260806 16:16:58.887496 14045 leader_election.cc:288] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:16:58.887676 14052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:16:58.888007 14052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [term 1 LEADER]: Becoming Leader. State: Replica: 3641097f8d954eaf9fbf075936b18b28, State: Running, Role: LEADER
I20260806 16:16:58.888429 14052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3641097f8d954eaf9fbf075936b18b28" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43743 } }
I20260806 16:16:58.889073 14045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:16:58.890419 14053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3641097f8d954eaf9fbf075936b18b28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3641097f8d954eaf9fbf075936b18b28" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43743 } } }
I20260806 16:16:58.890570 14053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [sys.catalog]: This master's current role is: LEADER
I20260806 16:16:58.890942 14054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3641097f8d954eaf9fbf075936b18b28. Latest consensus state: current_term: 1 leader_uuid: "3641097f8d954eaf9fbf075936b18b28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3641097f8d954eaf9fbf075936b18b28" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43743 } } }
I20260806 16:16:58.891062 14054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28 [sys.catalog]: This master's current role is: LEADER
I20260806 16:16:58.891348 14067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:16:58.894771 14067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:16:58.901978 14067 catalog_manager.cc:1347] Generated new cluster ID: 39c5427f196d4ecb961e4235cf164096
I20260806 16:16:58.902074 14067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:16:58.916484 14067 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:16:58.917783 14067 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:16:58.925800 14067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28: Generated new TSK 0
I20260806 16:16:58.926721 14067 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:16:58.967959 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43743
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:16:59.117079 14086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:16:59.117509 14086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:16:59.117571 14086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:16:59.122756 14086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:16:59.122867 14086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:16:59.122916 14086 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:16:59.122962 14086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:16:59.130220 14086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.245.190:43743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:16:59.132653 14086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:16:59.134619 14086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:16:59.148936 14099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:59.148936 14097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:59.148936 14102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:16:59.149446 14086 server_base.cc:1034] running on GCE node
I20260806 16:16:59.150079 14086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:16:59.151039 14086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:16:59.152235 14086 hybrid_clock.cc:648] HybridClock initialized: now 1786033019152175 us; error 74 us; skew 500 ppm
Aug 06 16:16:59 dist-test-slave-t3q3 krb5kdc[13907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033019, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:16:59.156234 14086 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:16:59.157575 14086 webserver.cc:460] Webserver started at http://127.11.245.129:40147/ using document root <none> and password file <none>
I20260806 16:16:59.158378 14086 fs_manager.cc:359] Metadata directory not provided
I20260806 16:16:59.158483 14086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:16:59.158789 14086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:16:59.161897 14086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "bdf8f17616a54f409d70f72f82b2a82f"
format_stamp: "Formatted at 2026-08-06 16:16:59 on dist-test-slave-t3q3"
server_key: "69330d13b54b3a4efd882d35523d36e8"
server_key_iv: "e45456cd43d08eacc076592d4eca5635"
server_key_version: "encryptionkey@0"
I20260806 16:16:59.162748 14086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "bdf8f17616a54f409d70f72f82b2a82f"
format_stamp: "Formatted at 2026-08-06 16:16:59 on dist-test-slave-t3q3"
server_key: "69330d13b54b3a4efd882d35523d36e8"
server_key_iv: "e45456cd43d08eacc076592d4eca5635"
server_key_version: "encryptionkey@0"
I20260806 16:16:59.167804 14086 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:16:59.171288 14114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:59.173310 14086 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:16:59.173540 14086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "bdf8f17616a54f409d70f72f82b2a82f"
format_stamp: "Formatted at 2026-08-06 16:16:59 on dist-test-slave-t3q3"
server_key: "69330d13b54b3a4efd882d35523d36e8"
server_key_iv: "e45456cd43d08eacc076592d4eca5635"
server_key_version: "encryptionkey@0"
I20260806 16:16:59.173844 14086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:16:59.186343 14086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:16:59.190402 14086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:16:59.190752 14086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:16:59.192868 14086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:16:59.192950 14086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:59.193063 14086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:16:59.193106 14086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 16:16:59 dist-test-slave-t3q3 krb5kdc[13907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033019, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:16:59.206132 14086 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:45401
I20260806 16:16:59.206427 14294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:45401 every 8 connection(s)
I20260806 16:16:59.207547 14086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:16:59.209384 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 14086
I20260806 16:16:59.209537 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:16:59.209694 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.193804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:39443 (local address 127.11.245.190:43743)
0806 16:16:59.194011 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:59.194017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:59.195682 (+ 1665us) server_negotiation.cc:407] Connection header received
0806 16:16:59.197107 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:59.197114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:59.197213 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:59.197318 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:59.198895 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.199890 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.201321 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.201604 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.201745 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.205026 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:59.205031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:59.205034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:59.205074 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:59.207048 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:59.207679 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:59.207683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:59.207686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:59.207788 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:59.208246 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:59.208250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:59.208252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:59.208570 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:59.208703 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:59.209293 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 16:16:59.209466 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 16:16:59.209933 12246 external_mini_cluster.cc:1428] Setting key 431927399f611064d7a2071f78171cc2
I20260806 16:16:59.210659 14127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.194894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43743 (local address 127.11.245.129:39443)
0806 16:16:59.195483 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:59.195544 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:59.196833 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:59.197545 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:59.197554 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:59.198084 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:16:59.198695 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.198712 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.200037 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.200041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.201174 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.201185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.201819 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.201824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.202027 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.202760 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:16:59.202778 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:16:59.204805 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:16:59.207208 (+ 2403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:59.207215 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:59.207234 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:59.207547 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:59.207896 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:59.207899 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:59.207902 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:59.208114 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:59.208704 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:16:59.208714 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:16:59.209006 (+ 292us) client_negotiation.cc:769] Sending connection context
0806 16:16:59.209189 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 16:16:59.209544 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":178,"threads_started":1}
I20260806 16:16:59.212381 14295 heartbeater.cc:347] Connected to a master server at 127.11.245.190:43743
I20260806 16:16:59.212610 14295 heartbeater.cc:464] Registering TS with master...
I20260806 16:16:59.213302 14295 heartbeater.cc:511] Master 127.11.245.190:43743 requested a full tablet report, sending...
I20260806 16:16:59.216361 13977 ts_manager.cc:194] Registered new tserver with Master: bdf8f17616a54f409d70f72f82b2a82f (127.11.245.129:45401)
I20260806 16:16:59.217929 13977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:39443
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:16:59.227886 14308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.216048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43743 (local address 127.11.245.129:33369)
0806 16:16:59.216473 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:59.216488 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:59.216618 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:59.218629 (+ 2011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:59.218632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:59.218911 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:16:59.219043 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.219050 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.220202 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.220209 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.221290 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.221306 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.222085 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.222088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.222174 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.223074 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:16:59.223096 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:16:59.223605 (+ 509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:16:59.226099 (+ 2494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:59.226102 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:59.226104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:59.226372 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:59.226724 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:59.226727 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:59.226728 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:59.226823 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:59.227347 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:16:59.227349 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:16:59.227464 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 16:16:59.227558 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 16:16:59.227700 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":4,"thread_start_us":149,"threads_started":1}
I20260806 16:16:59.227883 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.218191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:33369 (local address 127.11.245.190:43743)
0806 16:16:59.218370 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:59.218374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:59.218389 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:16:59.218437 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:59.218440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:59.218481 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:59.218602 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:59.219174 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.220007 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.221465 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.221912 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.222056 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.223851 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:59.223857 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:59.223862 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:59.223895 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:59.225929 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:59.226487 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:59.226493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:59.226497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:59.226600 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:59.226934 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:59.226949 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:59.226952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:59.227239 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:59.227354 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:59.227624 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 16:16:59.227750 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:16:59.252285 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43743
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:16:59.402645 14312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:16:59.403031 14312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:16:59.403090 14312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:16:59.408329 14312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:16:59.408439 14312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:16:59.408489 14312 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:16:59.408596 14312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:16:59.415969 14312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.245.190:43743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:16:59.417717 14312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:16:59.419171 14312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:16:59.432248 14321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:59.432312 14324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:59.432248 14320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:16:59.432643 14312 server_base.cc:1034] running on GCE node
I20260806 16:16:59.433382 14312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:16:59.434280 14312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:16:59.435527 14312 hybrid_clock.cc:648] HybridClock initialized: now 1786033019435489 us; error 58 us; skew 500 ppm
Aug 06 16:16:59 dist-test-slave-t3q3 krb5kdc[13907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033019, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:16:59.439371 14312 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:16:59.440809 14312 webserver.cc:460] Webserver started at http://127.11.245.130:38595/ using document root <none> and password file <none>
I20260806 16:16:59.441599 14312 fs_manager.cc:359] Metadata directory not provided
I20260806 16:16:59.441707 14312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:16:59.442016 14312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:16:59.445122 14312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "a21f64bbe6184d0585356b0b6e0f571b"
format_stamp: "Formatted at 2026-08-06 16:16:59 on dist-test-slave-t3q3"
server_key: "b0fecd00e1200d8f9e3bdf01c5139c4f"
server_key_iv: "da0f33fb3a090de4868ff72cfdd4eef9"
server_key_version: "encryptionkey@0"
I20260806 16:16:59.445904 14312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "a21f64bbe6184d0585356b0b6e0f571b"
format_stamp: "Formatted at 2026-08-06 16:16:59 on dist-test-slave-t3q3"
server_key: "b0fecd00e1200d8f9e3bdf01c5139c4f"
server_key_iv: "da0f33fb3a090de4868ff72cfdd4eef9"
server_key_version: "encryptionkey@0"
I20260806 16:16:59.451467 14312 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:16:59.455284 14332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:59.457211 14312 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:16:59.457407 14312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "a21f64bbe6184d0585356b0b6e0f571b"
format_stamp: "Formatted at 2026-08-06 16:16:59 on dist-test-slave-t3q3"
server_key: "b0fecd00e1200d8f9e3bdf01c5139c4f"
server_key_iv: "da0f33fb3a090de4868ff72cfdd4eef9"
server_key_version: "encryptionkey@0"
I20260806 16:16:59.457571 14312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:16:59.490736 14312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:16:59.496277 14312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:16:59.496682 14312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:16:59.498912 14312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:16:59.499017 14312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:59.499099 14312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:16:59.499145 14312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 16:16:59 dist-test-slave-t3q3 krb5kdc[13907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033019, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:16:59.513259 14312 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:41367
I20260806 16:16:59.513656 14512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:41367 every 8 connection(s)
I20260806 16:16:59.514919 14312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:16:59.517737 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.500770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:48405 (local address 127.11.245.190:43743)
0806 16:16:59.500985 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:59.500991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:59.502322 (+ 1331us) server_negotiation.cc:407] Connection header received
0806 16:16:59.504497 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:59.504507 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:59.504607 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:59.504747 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:59.506422 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.507209 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.508979 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.509237 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.509389 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.512762 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:59.512766 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:59.512769 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:59.512817 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:59.514703 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:59.515507 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:59.515511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:59.515513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:59.515663 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:59.516215 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:59.516218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:59.516220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:59.516491 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:59.516594 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:59.517431 (+ 837us) server_negotiation.cc:299] Negotiation successful
0806 16:16:59.517577 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 16:16:59.518829 14347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.501353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43743 (local address 127.11.245.130:48405)
0806 16:16:59.502069 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:59.502154 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:59.504022 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:59.505021 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:59.505030 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:59.505527 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:16:59.506234 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.506251 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.507498 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.507502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.508814 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.508828 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.509449 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.509455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.509716 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.510527 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:16:59.510547 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:16:59.512532 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:16:59.514911 (+ 2379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:59.514919 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:59.514938 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:59.515357 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:59.515788 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:59.515791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:59.515794 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:59.515998 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:59.516733 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:16:59.516740 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:16:59.517091 (+ 351us) client_negotiation.cc:769] Sending connection context
0806 16:16:59.517291 (+ 200us) client_negotiation.cc:240] Negotiation successful
0806 16:16:59.517654 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":149,"threads_started":1}
I20260806 16:16:59.520529 14516 heartbeater.cc:347] Connected to a master server at 127.11.245.190:43743
I20260806 16:16:59.520853 14516 heartbeater.cc:464] Registering TS with master...
I20260806 16:16:59.521672 14516 heartbeater.cc:511] Master 127.11.245.190:43743 requested a full tablet report, sending...
I20260806 16:16:59.523460 13978 ts_manager.cc:194] Registered new tserver with Master: a21f64bbe6184d0585356b0b6e0f571b (127.11.245.130:41367)
I20260806 16:16:59.523949 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 14312
I20260806 16:16:59.524108 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:16:59.524533 12246 external_mini_cluster.cc:1428] Setting key 9ad4e72acb0a27a5b411f52bef39b665
I20260806 16:16:59.524616 13978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:48405
I20260806 16:16:59.533646 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.524399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:55133 (local address 127.11.245.190:43743)
0806 16:16:59.524535 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:59.524540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:59.524558 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:16:59.524775 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:59.524779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:59.524828 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:59.524880 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:59.525703 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.526481 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.527843 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.528183 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.528296 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.529770 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:59.529777 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:59.529781 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:59.529816 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:59.531628 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:59.532171 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:59.532176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:59.532178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:59.532269 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:59.532591 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:59.532594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:59.532596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:59.532876 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:59.532999 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:59.533300 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 16:16:59.533437 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 16:16:59.533656 14525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.523941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43743 (local address 127.11.245.130:55133)
0806 16:16:59.524402 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:59.524421 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:59.524629 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:59.525081 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:59.525083 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:59.525401 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:16:59.525569 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.525579 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.526641 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.526646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.527684 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.527701 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.528323 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.528330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.528475 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.529170 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:16:59.529185 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:16:59.529606 (+ 421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:16:59.531757 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:59.531762 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:59.531764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:59.532010 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:59.532373 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:59.532377 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:59.532379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:59.532478 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:59.532991 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:16:59.532994 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:16:59.533125 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 16:16:59.533241 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 16:16:59.533444 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":207,"threads_started":1}
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:16:59.564925 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43743
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:16:59.721971 14530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:16:59.722388 14530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:16:59.722489 14530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:16:59.726884 14530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:16:59.726961 14530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:16:59.727006 14530 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:16:59.727080 14530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:16:59.733438 14530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.245.190:43743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:16:59.734989 14530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:16:59.737164 14530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:16:59.750699 14542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:16:59.750718 14539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:16:59.750880 14530 server_base.cc:1034] running on GCE node
W20260806 16:16:59.750699 14546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:16:59.751760 14530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:16:59.752784 14530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:16:59.753991 14530 hybrid_clock.cc:648] HybridClock initialized: now 1786033019753975 us; error 34 us; skew 500 ppm
Aug 06 16:16:59 dist-test-slave-t3q3 krb5kdc[13907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033019, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:16:59.757695 14530 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:16:59.758983 14530 webserver.cc:460] Webserver started at http://127.11.245.131:40049/ using document root <none> and password file <none>
I20260806 16:16:59.759873 14530 fs_manager.cc:359] Metadata directory not provided
I20260806 16:16:59.759958 14530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:16:59.760291 14530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:16:59.763077 14530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "f06d38a4e09f4651a7c73ccd8676f4ad"
format_stamp: "Formatted at 2026-08-06 16:16:59 on dist-test-slave-t3q3"
server_key: "f63d76bbce933b7d05e97775eec93c0c"
server_key_iv: "8f753a1ed5703304469e9a1d6c5688e1"
server_key_version: "encryptionkey@0"
I20260806 16:16:59.763837 14530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "f06d38a4e09f4651a7c73ccd8676f4ad"
format_stamp: "Formatted at 2026-08-06 16:16:59 on dist-test-slave-t3q3"
server_key: "f63d76bbce933b7d05e97775eec93c0c"
server_key_iv: "8f753a1ed5703304469e9a1d6c5688e1"
server_key_version: "encryptionkey@0"
I20260806 16:16:59.769203 14530 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:16:59.772802 14554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:59.774614 14530 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:16:59.774758 14530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "f06d38a4e09f4651a7c73ccd8676f4ad"
format_stamp: "Formatted at 2026-08-06 16:16:59 on dist-test-slave-t3q3"
server_key: "f63d76bbce933b7d05e97775eec93c0c"
server_key_iv: "8f753a1ed5703304469e9a1d6c5688e1"
server_key_version: "encryptionkey@0"
I20260806 16:16:59.774892 14530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:16:59.801182 14530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:16:59.805408 14530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:16:59.805620 14530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:16:59.807518 14530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:16:59.807616 14530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:16:59.807699 14530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:16:59.807750 14530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 16:16:59 dist-test-slave-t3q3 krb5kdc[13907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033019, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:16:59.822515 14530 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:34311
I20260806 16:16:59.822554 14721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:34311 every 8 connection(s)
I20260806 16:16:59.823889 14530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:16:59.826845 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.808753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:53543 (local address 127.11.245.190:43743)
0806 16:16:59.808958 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:59.808965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:59.810208 (+ 1243us) server_negotiation.cc:407] Connection header received
0806 16:16:59.811515 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:59.811523 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:59.811622 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:59.811748 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:59.813594 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.814314 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.815764 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.816020 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.816198 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.821520 (+ 5322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:59.821524 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:59.821527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:59.821562 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:59.823613 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:59.824410 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:59.824415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:59.824417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:59.824559 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:59.825234 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:59.825238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:59.825240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:59.825692 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:59.825807 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:59.826518 (+ 711us) server_negotiation.cc:299] Negotiation successful
0806 16:16:59.826685 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 16:16:59.827039 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 14530
I20260806 16:16:59.827186 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:16:59.827538 12246 external_mini_cluster.cc:1428] Setting key dc175c91e4b911572fc35d5fc4e31626
I20260806 16:16:59.828274 14566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.809388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43743 (local address 127.11.245.131:53543)
0806 16:16:59.809983 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:59.810046 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:59.811193 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:59.812054 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:59.812067 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:59.812645 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:16:59.813371 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.813389 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.814474 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.814479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.815609 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.815622 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.816265 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.816269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.816534 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.817477 (+ 943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:16:59.817508 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:16:59.821235 (+ 3727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:16:59.823746 (+ 2511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:59.823754 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:59.823774 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:59.824244 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:59.824886 (+ 642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:59.824889 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:59.824892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:59.825094 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:59.825833 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:16:59.825840 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:16:59.826201 (+ 361us) client_negotiation.cc:769] Sending connection context
0806 16:16:59.826408 (+ 207us) client_negotiation.cc:240] Negotiation successful
0806 16:16:59.826843 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":175,"threads_started":1}
I20260806 16:16:59.830425 14723 heartbeater.cc:347] Connected to a master server at 127.11.245.190:43743
I20260806 16:16:59.830659 14723 heartbeater.cc:464] Registering TS with master...
I20260806 16:16:59.831350 14723 heartbeater.cc:511] Master 127.11.245.190:43743 requested a full tablet report, sending...
I20260806 16:16:59.832892 13977 ts_manager.cc:194] Registered new tserver with Master: f06d38a4e09f4651a7c73ccd8676f4ad (127.11.245.131:34311)
I20260806 16:16:59.833829 13977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:53543
I20260806 16:16:59.841915 14733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.833756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43743 (local address 127.11.245.131:42831)
0806 16:16:59.834150 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:59.834164 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:59.834313 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:59.834628 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:59.834632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:59.834900 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:16:59.835022 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.835029 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.835834 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.835838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.836541 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.836552 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.837045 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.837049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.837149 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.837824 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:16:59.837836 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:16:59.838252 (+ 416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:16:59.840215 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:59.840219 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:59.840221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:59.840451 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:59.840756 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:16:59.840759 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:16:59.840761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:16:59.840856 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:16:59.841266 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:16:59.841270 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:16:59.841388 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 16:16:59.841500 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 16:16:59.841682 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":190,"threads_started":1}
I20260806 16:16:59.841903 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.833778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:42831 (local address 127.11.245.190:43743)
0806 16:16:59.833921 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:59.833926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:59.834277 (+ 351us) server_negotiation.cc:407] Connection header received
0806 16:16:59.834466 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:59.834468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:59.834513 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:59.834606 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:59.835145 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.835697 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.836671 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.836918 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.837037 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.838401 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:59.838405 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:59.838408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:59.838435 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:59.840004 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:59.840565 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:59.840569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:59.840571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:59.840658 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:59.840951 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:59.840956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:59.840958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:59.841163 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:59.841279 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:59.841559 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 16:16:59.841685 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 16:16:59.846307 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 16:16:59 dist-test-slave-t3q3 krb5kdc[13907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033019, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 16:16:59 dist-test-slave-t3q3 krb5kdc[13907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033019, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:16:59.885514 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.875216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33558 (local address 127.11.245.190:43743)
0806 16:16:59.875413 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:59.875419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:59.875556 (+ 137us) server_negotiation.cc:407] Connection header received
0806 16:16:59.875761 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:59.875766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:59.875864 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:59.875983 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:16:59.876901 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.877486 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.878394 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.878683 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.878802 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.881947 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:16:59.881951 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:16:59.881953 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:16:59.881982 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:16:59.883529 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:59.884036 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:59.884041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:59.884044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:59.884188 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:16:59.884492 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:16:59.884496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:16:59.884498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:16:59.884728 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:16:59.884890 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:59.885154 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 16:16:59.885308 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 16:16:59.889683 13977 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:16:59.894084 13977 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:16:59.913797 14752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.906418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37488 (local address 127.11.245.130:41367)
0806 16:16:59.907120 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:59.907135 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:59.907400 (+ 265us) server_negotiation.cc:407] Connection header received
0806 16:16:59.908003 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:59.908008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:59.908242 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:59.908417 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:16:59.908698 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.910059 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.911919 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.912669 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.912863 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.913141 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:59.913433 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 16:16:59.913581 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":155,"threads_started":1}
I20260806 16:16:59.913877 14747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.906317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59188 (local address 127.11.245.129:45401)
0806 16:16:59.906814 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:59.906823 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:59.907244 (+ 421us) server_negotiation.cc:407] Connection header received
0806 16:16:59.907568 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:59.907582 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:59.907864 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:59.908022 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:16:59.908738 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.909745 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.911864 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.912843 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.913037 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.913184 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:59.913576 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 16:16:59.913716 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":99,"threads_started":1}
I20260806 16:16:59.914026 14753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.907112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:34311 (local address 127.0.0.1:58748)
0806 16:16:59.907549 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:59.907564 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:59.907666 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:59.908592 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:59.908597 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:59.908619 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:16:59.908739 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.908746 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.910567 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.910570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.911489 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.911500 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.913288 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.913291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.913610 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.913632 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 16:16:59.913699 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 16:16:59.913864 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":252,"thread_start_us":112,"threads_started":1}
I20260806 16:16:59.914304 14750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.906078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:41367 (local address 127.0.0.1:37488)
0806 16:16:59.907065 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:59.907129 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:59.907303 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:59.908386 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:59.908389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:59.908407 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:16:59.908552 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.908563 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.910190 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.910193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.911713 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.911725 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.912853 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.912857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.913084 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.913186 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 16:16:59.914097 (+ 911us) client_negotiation.cc:240] Negotiation successful
0806 16:16:59.914192 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":96,"thread_start_us":151,"threads_started":1}
I20260806 16:16:59.914552 14755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.907465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58748 (local address 127.11.245.131:34311)
0806 16:16:59.907887 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 16:16:59.907901 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:16:59.907934 (+ 33us) server_negotiation.cc:407] Connection header received
0806 16:16:59.908034 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:16:59.908038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:16:59.908267 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:16:59.908424 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:16:59.908877 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.910429 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.911631 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.912778 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.912985 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.913473 (+ 488us) server_negotiation.cc:1035] Waiting for connection context
0806 16:16:59.914239 (+ 766us) server_negotiation.cc:299] Negotiation successful
0806 16:16:59.914371 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":156,"threads_started":1}
I20260806 16:16:59.915067 14746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:16:59.906077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:45401 (local address 127.0.0.1:59188)
0806 16:16:59.907065 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0806 16:16:59.907131 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 16:16:59.907332 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:16:59.908301 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:16:59.908308 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:16:59.908339 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:16:59.908554 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.908565 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.909895 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.909898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.911713 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:16:59.911725 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:16:59.913263 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:16:59.913269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:16:59.913395 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:16:59.913418 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 16:16:59.914860 (+ 1442us) client_negotiation.cc:240] Negotiation successful
0806 16:16:59.914953 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"mutex_wait_us":475,"thread_start_us":142,"threads_started":1}
I20260806 16:16:59.918756 14624 tablet_service.cc:1467] Processing CreateTablet for tablet 87f280a07baf47baaffc0e67cc89642a (DEFAULT_TABLE table=test-table [id=97c8714d72b5464fb19e794ebec04670]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:16:59.918756 14412 tablet_service.cc:1467] Processing CreateTablet for tablet 87f280a07baf47baaffc0e67cc89642a (DEFAULT_TABLE table=test-table [id=97c8714d72b5464fb19e794ebec04670]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:16:59.918761 14194 tablet_service.cc:1467] Processing CreateTablet for tablet 87f280a07baf47baaffc0e67cc89642a (DEFAULT_TABLE table=test-table [id=97c8714d72b5464fb19e794ebec04670]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:16:59.920413 14624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87f280a07baf47baaffc0e67cc89642a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:16:59.920413 14412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87f280a07baf47baaffc0e67cc89642a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:16:59.920776 14194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87f280a07baf47baaffc0e67cc89642a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:16:59.929884 14756 tablet_bootstrap.cc:492] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad: Bootstrap starting.
I20260806 16:16:59.930675 14759 tablet_bootstrap.cc:492] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b: Bootstrap starting.
I20260806 16:16:59.931617 14758 tablet_bootstrap.cc:492] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f: Bootstrap starting.
I20260806 16:16:59.933911 14758 tablet_bootstrap.cc:654] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f: Neither blocks nor log segments found. Creating new log.
I20260806 16:16:59.933913 14759 tablet_bootstrap.cc:654] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b: Neither blocks nor log segments found. Creating new log.
I20260806 16:16:59.934917 14758 log.cc:824] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f: Log is configured to *not* fsync() on all Append() calls
I20260806 16:16:59.934937 14759 log.cc:824] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b: Log is configured to *not* fsync() on all Append() calls
I20260806 16:16:59.936870 14756 tablet_bootstrap.cc:654] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad: Neither blocks nor log segments found. Creating new log.
I20260806 16:16:59.937525 14758 tablet_bootstrap.cc:492] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f: No bootstrap required, opened a new log
I20260806 16:16:59.937538 14759 tablet_bootstrap.cc:492] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b: No bootstrap required, opened a new log
I20260806 16:16:59.937786 14758 ts_tablet_manager.cc:1397] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260806 16:16:59.937781 14759 ts_tablet_manager.cc:1397] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 16:16:59.937955 14756 log.cc:824] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad: Log is configured to *not* fsync() on all Append() calls
I20260806 16:16:59.940644 14756 tablet_bootstrap.cc:492] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad: No bootstrap required, opened a new log
I20260806 16:16:59.940892 14756 ts_tablet_manager.cc:1397] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260806 16:16:59.942119 14758 raft_consensus.cc:348] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:16:59.942441 14758 raft_consensus.cc:374] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:16:59.942516 14758 raft_consensus.cc:729] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf8f17616a54f409d70f72f82b2a82f, State: Initialized, Role: FOLLOWER
I20260806 16:16:59.942420 14759 raft_consensus.cc:348] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:16:59.942695 14759 raft_consensus.cc:374] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:16:59.942767 14759 raft_consensus.cc:729] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a21f64bbe6184d0585356b0b6e0f571b, State: Initialized, Role: FOLLOWER
I20260806 16:16:59.943161 14758 consensus_queue.cc:260] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:16:59.943234 14759 consensus_queue.cc:260] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:16:59.943994 14295 heartbeater.cc:503] Master 127.11.245.190:43743 was elected leader, sending a full tablet report...
I20260806 16:16:59.944428 14758 ts_tablet_manager.cc:1428] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260806 16:16:59.944443 14759 ts_tablet_manager.cc:1428] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260806 16:16:59.944941 14516 heartbeater.cc:503] Master 127.11.245.190:43743 was elected leader, sending a full tablet report...
I20260806 16:16:59.945343 14756 raft_consensus.cc:348] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:16:59.945636 14756 raft_consensus.cc:374] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:16:59.945705 14756 raft_consensus.cc:729] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f06d38a4e09f4651a7c73ccd8676f4ad, State: Initialized, Role: FOLLOWER
I20260806 16:16:59.946239 14756 consensus_queue.cc:260] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:16:59.947034 14723 heartbeater.cc:503] Master 127.11.245.190:43743 was elected leader, sending a full tablet report...
I20260806 16:16:59.948313 14756 ts_tablet_manager.cc:1428] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
W20260806 16:16:59.959453 14296 tablet.cc:2369] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:17:00.016819 14517 tablet.cc:2369] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:17:00.075366 14724 tablet.cc:2369] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:17:00.209416 14769 raft_consensus.cc:482] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:00.209745 14769 raft_consensus.cc:504] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.211611 14769 leader_election.cc:288] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf8f17616a54f409d70f72f82b2a82f (127.11.245.129:45401), f06d38a4e09f4651a7c73ccd8676f4ad (127.11.245.131:34311)
I20260806 16:17:00.217890 14755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:00.212837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:55781 (local address 127.11.245.131:34311)
0806 16:17:00.213010 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:00.213018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:00.213051 (+ 33us) server_negotiation.cc:407] Connection header received
0806 16:17:00.213258 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:00.213262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:00.213334 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:00.213437 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:00.213824 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.214694 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.216362 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.217059 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.217176 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:00.217228 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:00.217632 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 16:17:00.217731 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 16:17:00.217898 14776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:00.212537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:34311 (local address 127.11.245.130:55781)
0806 16:17:00.212938 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:00.212953 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:00.213069 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:00.213458 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:00.213461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:00.213478 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:00.213648 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:00.213653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.214840 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.214844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:00.216238 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:00.216250 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.217298 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.217302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:00.217474 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:00.217500 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 16:17:00.217605 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 16:17:00.217709 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":129,"thread_start_us":110,"threads_started":1}
I20260806 16:17:00.218073 14747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:00.212180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:59771 (local address 127.11.245.129:45401)
0806 16:17:00.212377 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:00.212384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:00.212588 (+ 204us) server_negotiation.cc:407] Connection header received
0806 16:17:00.212895 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:00.212899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:00.212970 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:00.213054 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:00.213757 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.214694 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.216299 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.216962 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.217138 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:00.217215 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:00.217821 (+ 606us) server_negotiation.cc:299] Negotiation successful
0806 16:17:00.217927 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 16:17:00.218302 14773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:00.211985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:45401 (local address 127.11.245.130:59771)
0806 16:17:00.212433 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:00.212459 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:00.212711 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:00.213349 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:00.213353 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:00.213381 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:00.213611 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:00.213620 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.214895 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.214898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:00.216138 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:00.216152 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.217138 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.217142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:00.217347 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:00.217540 (+ 193us) client_negotiation.cc:769] Sending connection context
0806 16:17:00.217791 (+ 251us) client_negotiation.cc:240] Negotiation successful
0806 16:17:00.217922 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":5,"thread_start_us":171,"threads_started":1}
I20260806 16:17:00.218940 14222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87f280a07baf47baaffc0e67cc89642a" candidate_uuid: "a21f64bbe6184d0585356b0b6e0f571b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf8f17616a54f409d70f72f82b2a82f" is_pre_election: true
I20260806 16:17:00.218816 14652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87f280a07baf47baaffc0e67cc89642a" candidate_uuid: "a21f64bbe6184d0585356b0b6e0f571b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" is_pre_election: true
I20260806 16:17:00.219352 14222 raft_consensus.cc:2454] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a21f64bbe6184d0585356b0b6e0f571b in term 0.
I20260806 16:17:00.219359 14652 raft_consensus.cc:2454] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a21f64bbe6184d0585356b0b6e0f571b in term 0.
I20260806 16:17:00.220009 14333 leader_election.cc:302] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a21f64bbe6184d0585356b0b6e0f571b, f06d38a4e09f4651a7c73ccd8676f4ad; no voters:
I20260806 16:17:00.220539 14769 raft_consensus.cc:2781] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:00.220718 14769 raft_consensus.cc:482] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:00.220795 14769 raft_consensus.cc:3037] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:00.222472 14769 raft_consensus.cc:504] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.223045 14769 leader_election.cc:288] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [CANDIDATE]: Term 1 election: Requested vote from peers bdf8f17616a54f409d70f72f82b2a82f (127.11.245.129:45401), f06d38a4e09f4651a7c73ccd8676f4ad (127.11.245.131:34311)
I20260806 16:17:00.223753 14222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87f280a07baf47baaffc0e67cc89642a" candidate_uuid: "a21f64bbe6184d0585356b0b6e0f571b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf8f17616a54f409d70f72f82b2a82f"
I20260806 16:17:00.223848 14652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87f280a07baf47baaffc0e67cc89642a" candidate_uuid: "a21f64bbe6184d0585356b0b6e0f571b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad"
I20260806 16:17:00.223954 14222 raft_consensus.cc:3037] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:00.224038 14652 raft_consensus.cc:3037] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:00.225458 14222 raft_consensus.cc:2454] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a21f64bbe6184d0585356b0b6e0f571b in term 1.
I20260806 16:17:00.225921 14333 leader_election.cc:302] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a21f64bbe6184d0585356b0b6e0f571b, bdf8f17616a54f409d70f72f82b2a82f; no voters:
I20260806 16:17:00.225971 14652 raft_consensus.cc:2454] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a21f64bbe6184d0585356b0b6e0f571b in term 1.
I20260806 16:17:00.226158 14769 raft_consensus.cc:2781] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:00.226449 14769 raft_consensus.cc:686] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [term 1 LEADER]: Becoming Leader. State: Replica: a21f64bbe6184d0585356b0b6e0f571b, State: Running, Role: LEADER
I20260806 16:17:00.226970 14769 consensus_queue.cc:237] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.231622 13977 catalog_manager.cc:5495] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b reported cstate change: term changed from 0 to 1, leader changed from <none> to a21f64bbe6184d0585356b0b6e0f571b (127.11.245.130). New cstate: current_term: 1 leader_uuid: "a21f64bbe6184d0585356b0b6e0f571b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } health_report { overall_health: UNKNOWN } } }
I20260806 16:17:00.297847 14752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:00.292601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37494 (local address 127.11.245.130:41367)
0806 16:17:00.292787 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:00.292791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:00.292809 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:17:00.293158 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:00.293161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:00.293256 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:00.293367 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:00.293859 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.294446 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.295480 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.295729 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.295832 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:00.296538 (+ 706us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:00.297184 (+ 646us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:00.297331 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:00.297491 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 16:17:00.297616 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 16:17:00.306573 14652 raft_consensus.cc:1261] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad [term 1 FOLLOWER]: Refusing update from remote peer a21f64bbe6184d0585356b0b6e0f571b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:00.306573 14222 raft_consensus.cc:1261] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f [term 1 FOLLOWER]: Refusing update from remote peer a21f64bbe6184d0585356b0b6e0f571b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:00.307657 14779 consensus_queue.cc:1035] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:00.307974 14769 consensus_queue.cc:1035] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:00.317516 14784 mvcc.cc:205] Tried to move back new op lower bound from 7315591251163840512 to 7315591250853474304. Current Snapshot: MvccSnapshot[applied={T|T < 7315591251163840512}]
I20260806 16:17:00.318693 14786 mvcc.cc:205] Tried to move back new op lower bound from 7315591251163840512 to 7315591250853474304. Current Snapshot: MvccSnapshot[applied={T|T < 7315591251163840512}]
I20260806 16:17:00.322479 14787 mvcc.cc:205] Tried to move back new op lower bound from 7315591251163840512 to 7315591250853474304. Current Snapshot: MvccSnapshot[applied={T|T < 7315591251163840512}]
I20260806 16:17:00.334185 13977 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:00.334522 13977 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:00.338894 13977 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28: Sending DeleteTablet for 3 replicas of tablet 87f280a07baf47baaffc0e67cc89642a
I20260806 16:17:00.340384 14194 tablet_service.cc:1514] Processing DeleteTablet for tablet 87f280a07baf47baaffc0e67cc89642a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:00 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:59188
I20260806 16:17:00.340401 14624 tablet_service.cc:1514] Processing DeleteTablet for tablet 87f280a07baf47baaffc0e67cc89642a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:00 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:58748
I20260806 16:17:00.340390 14412 tablet_service.cc:1514] Processing DeleteTablet for tablet 87f280a07baf47baaffc0e67cc89642a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:00 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:37488
I20260806 16:17:00.341135 14801 tablet_replica.cc:330] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b: stopping tablet replica
I20260806 16:17:00.341141 14797 tablet_replica.cc:330] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad: stopping tablet replica
I20260806 16:17:00.341630 14801 raft_consensus.cc:2229] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:17:00.341629 14797 raft_consensus.cc:2229] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:00.342196 14797 raft_consensus.cc:2258] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:00.342346 14801 raft_consensus.cc:2258] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:00.342878 14799 tablet_replica.cc:330] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f: stopping tablet replica
I20260806 16:17:00.343266 14799 raft_consensus.cc:2229] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:00.342491 13977 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 16:17:00.343696 14799 raft_consensus.cc:2258] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 16:17:00.343812 13977 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:17:00.346118 14797 ts_tablet_manager.cc:1905] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:00.346118 14801 ts_tablet_manager.cc:1905] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:00.350600 14194 tablet_service.cc:1467] Processing CreateTablet for tablet b7b7fb70dc7746e59c53078145fa0685 (DEFAULT_TABLE table=test-table [id=60d16fa3ee034a0a9cd2d6462c265f66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:00.350752 14412 tablet_service.cc:1467] Processing CreateTablet for tablet b7b7fb70dc7746e59c53078145fa0685 (DEFAULT_TABLE table=test-table [id=60d16fa3ee034a0a9cd2d6462c265f66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:00.351027 14194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b7fb70dc7746e59c53078145fa0685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:00.351068 14412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b7fb70dc7746e59c53078145fa0685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:00.351251 14797 ts_tablet_manager.cc:1918] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:00.351384 14797 log.cc:1196] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/87f280a07baf47baaffc0e67cc89642a
I20260806 16:17:00.351792 14797 ts_tablet_manager.cc:1939] T 87f280a07baf47baaffc0e67cc89642a P f06d38a4e09f4651a7c73ccd8676f4ad: Deleting consensus metadata
I20260806 16:17:00.353945 14758 tablet_bootstrap.cc:492] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f: Bootstrap starting.
I20260806 16:17:00.354130 13956 catalog_manager.cc:4841] TS f06d38a4e09f4651a7c73ccd8676f4ad (127.11.245.131:34311): tablet 87f280a07baf47baaffc0e67cc89642a (table test-table [id=97c8714d72b5464fb19e794ebec04670]) successfully deleted
I20260806 16:17:00.355219 14758 tablet_bootstrap.cc:654] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:00.355216 14624 tablet_service.cc:1467] Processing CreateTablet for tablet b7b7fb70dc7746e59c53078145fa0685 (DEFAULT_TABLE table=test-table [id=60d16fa3ee034a0a9cd2d6462c265f66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:00.355549 14624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b7fb70dc7746e59c53078145fa0685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:00.355608 14759 tablet_bootstrap.cc:492] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b: Bootstrap starting.
I20260806 16:17:00.356938 14759 tablet_bootstrap.cc:654] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:00.358624 14756 tablet_bootstrap.cc:492] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad: Bootstrap starting.
I20260806 16:17:00.359036 14759 tablet_bootstrap.cc:492] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b: No bootstrap required, opened a new log
I20260806 16:17:00.359119 14759 ts_tablet_manager.cc:1397] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:17:00.359648 14759 raft_consensus.cc:348] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.359817 14759 raft_consensus.cc:374] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:00.359853 14759 raft_consensus.cc:729] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a21f64bbe6184d0585356b0b6e0f571b, State: Initialized, Role: FOLLOWER
I20260806 16:17:00.359936 14756 tablet_bootstrap.cc:654] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:00.360050 14759 consensus_queue.cc:260] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.360422 14759 ts_tablet_manager.cc:1428] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:17:00.362321 14756 tablet_bootstrap.cc:492] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad: No bootstrap required, opened a new log
I20260806 16:17:00.362470 14756 ts_tablet_manager.cc:1397] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 16:17:00.362912 14758 tablet_bootstrap.cc:492] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f: No bootstrap required, opened a new log
I20260806 16:17:00.363030 14758 ts_tablet_manager.cc:1397] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260806 16:17:00.363145 14756 raft_consensus.cc:348] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.363291 14756 raft_consensus.cc:374] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:00.363324 14756 raft_consensus.cc:729] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f06d38a4e09f4651a7c73ccd8676f4ad, State: Initialized, Role: FOLLOWER
I20260806 16:17:00.363493 14756 consensus_queue.cc:260] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.363641 14758 raft_consensus.cc:348] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.363775 14758 raft_consensus.cc:374] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:00.363808 14758 raft_consensus.cc:729] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf8f17616a54f409d70f72f82b2a82f, State: Initialized, Role: FOLLOWER
I20260806 16:17:00.363896 14756 ts_tablet_manager.cc:1428] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:17:00.363960 14758 consensus_queue.cc:260] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.364356 14758 ts_tablet_manager.cc:1428] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 16:17:00.369362 14799 ts_tablet_manager.cc:1905] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:00.369536 14801 ts_tablet_manager.cc:1918] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:00.369697 14801 log.cc:1196] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/87f280a07baf47baaffc0e67cc89642a
I20260806 16:17:00.370136 14801 ts_tablet_manager.cc:1939] T 87f280a07baf47baaffc0e67cc89642a P a21f64bbe6184d0585356b0b6e0f571b: Deleting consensus metadata
I20260806 16:17:00.371964 13960 catalog_manager.cc:4841] TS a21f64bbe6184d0585356b0b6e0f571b (127.11.245.130:41367): tablet 87f280a07baf47baaffc0e67cc89642a (table test-table [id=97c8714d72b5464fb19e794ebec04670]) successfully deleted
I20260806 16:17:00.372699 14799 ts_tablet_manager.cc:1918] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:00.372865 14799 log.cc:1196] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/wal/wals/87f280a07baf47baaffc0e67cc89642a
I20260806 16:17:00.373276 14799 ts_tablet_manager.cc:1939] T 87f280a07baf47baaffc0e67cc89642a P bdf8f17616a54f409d70f72f82b2a82f: Deleting consensus metadata
I20260806 16:17:00.374826 13956 catalog_manager.cc:4841] TS bdf8f17616a54f409d70f72f82b2a82f (127.11.245.129:45401): tablet 87f280a07baf47baaffc0e67cc89642a (table test-table [id=97c8714d72b5464fb19e794ebec04670]) successfully deleted
I20260806 16:17:00.644695 14772 raft_consensus.cc:482] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:00.644920 14772 raft_consensus.cc:504] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.646520 14772 leader_election.cc:288] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf8f17616a54f409d70f72f82b2a82f (127.11.245.129:45401), a21f64bbe6184d0585356b0b6e0f571b (127.11.245.130:41367)
I20260806 16:17:00.651464 14747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:00.647223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:34015 (local address 127.11.245.129:45401)
0806 16:17:00.647382 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:00.647387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:00.647513 (+ 126us) server_negotiation.cc:407] Connection header received
0806 16:17:00.647741 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:00.647744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:00.647794 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:00.647880 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:00.648342 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.648899 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.650210 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.650673 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.650830 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:00.650907 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:00.651194 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 16:17:00.651287 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 16:17:00.651458 14752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:00.647060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:33787 (local address 127.11.245.130:41367)
0806 16:17:00.647258 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:00.647264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:00.647573 (+ 309us) server_negotiation.cc:407] Connection header received
0806 16:17:00.647732 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:00.647735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:00.647803 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:00.647893 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:00.648340 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.648899 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.650227 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.650681 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.650836 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:00.650906 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:00.651194 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 16:17:00.651287 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 16:17:00.651476 14807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:00.646924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:41367 (local address 127.11.245.131:33787)
0806 16:17:00.647438 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:00.647450 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:00.647616 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:00.647929 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:00.647932 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:00.647948 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:00.648207 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:00.648214 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.649025 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.649030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:00.650083 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:00.650093 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.650846 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.650850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:00.651017 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:00.651070 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 16:17:00.651169 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 16:17:00.651278 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":313,"thread_start_us":60,"threads_started":1}
I20260806 16:17:00.651492 14806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:00.646924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:45401 (local address 127.11.245.131:34015)
0806 16:17:00.647388 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:00.647406 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:00.647600 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:00.647907 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:00.647910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:00.647928 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:00.648220 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:00.648227 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.649027 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.649031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:00.650073 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:00.650084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.650830 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.650834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:00.651020 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:00.651077 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 16:17:00.651171 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 16:17:00.651278 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":158,"spinlock_wait_cycles":29056,"thread_start_us":104,"threads_started":1}
I20260806 16:17:00.652474 14222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7b7fb70dc7746e59c53078145fa0685" candidate_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf8f17616a54f409d70f72f82b2a82f" is_pre_election: true
I20260806 16:17:00.652545 14439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7b7fb70dc7746e59c53078145fa0685" candidate_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a21f64bbe6184d0585356b0b6e0f571b" is_pre_election: true
I20260806 16:17:00.652734 14222 raft_consensus.cc:2454] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f06d38a4e09f4651a7c73ccd8676f4ad in term 0.
I20260806 16:17:00.652880 14439 raft_consensus.cc:2454] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f06d38a4e09f4651a7c73ccd8676f4ad in term 0.
I20260806 16:17:00.653280 14555 leader_election.cc:302] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf8f17616a54f409d70f72f82b2a82f, f06d38a4e09f4651a7c73ccd8676f4ad; no voters:
I20260806 16:17:00.653717 14772 raft_consensus.cc:2781] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:00.653784 14772 raft_consensus.cc:482] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:00.653813 14772 raft_consensus.cc:3037] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:00.655201 14772 raft_consensus.cc:504] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.655702 14772 leader_election.cc:288] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [CANDIDATE]: Term 1 election: Requested vote from peers bdf8f17616a54f409d70f72f82b2a82f (127.11.245.129:45401), a21f64bbe6184d0585356b0b6e0f571b (127.11.245.130:41367)
I20260806 16:17:00.656294 14439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7b7fb70dc7746e59c53078145fa0685" candidate_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a21f64bbe6184d0585356b0b6e0f571b"
I20260806 16:17:00.656457 14439 raft_consensus.cc:3037] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:00.656477 14222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7b7fb70dc7746e59c53078145fa0685" candidate_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf8f17616a54f409d70f72f82b2a82f"
I20260806 16:17:00.656680 14222 raft_consensus.cc:3037] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:00.657876 14222 raft_consensus.cc:2454] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f06d38a4e09f4651a7c73ccd8676f4ad in term 1.
I20260806 16:17:00.657881 14439 raft_consensus.cc:2454] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f06d38a4e09f4651a7c73ccd8676f4ad in term 1.
I20260806 16:17:00.658296 14555 leader_election.cc:302] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf8f17616a54f409d70f72f82b2a82f, f06d38a4e09f4651a7c73ccd8676f4ad; no voters:
I20260806 16:17:00.658560 14772 raft_consensus.cc:2781] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:00.658802 14772 raft_consensus.cc:686] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [term 1 LEADER]: Becoming Leader. State: Replica: f06d38a4e09f4651a7c73ccd8676f4ad, State: Running, Role: LEADER
I20260806 16:17:00.659421 14772 consensus_queue.cc:237] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.663117 13977 catalog_manager.cc:5495] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad reported cstate change: term changed from 0 to 1, leader changed from <none> to f06d38a4e09f4651a7c73ccd8676f4ad (127.11.245.131). New cstate: current_term: 1 leader_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } health_report { overall_health: HEALTHY } } }
Aug 06 16:17:00 dist-test-slave-t3q3 krb5kdc[13907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033018, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:00.777107 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:00.766712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43743 (local address 127.0.0.1:33572)
0806 16:17:00.767152 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:00.767170 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:00.767306 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:00.767697 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:00.767700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:00.768320 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:00.768447 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:00.768455 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.769344 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.769347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:00.769940 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:00.769949 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.770402 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.770405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:00.770468 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:00.771203 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:00.771216 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:00.773073 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:00.775152 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:00.775163 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:00.775172 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:00.775482 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:00.775837 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:00.775840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:00.775842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:00.775935 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:00.776507 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:00.776515 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:00.776648 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 16:17:00.776765 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 16:17:00.776923 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":168,"threads_started":1}
I20260806 16:17:00.777180 14817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:00.766856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33572 (local address 127.11.245.190:43743)
0806 16:17:00.767206 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:00.767215 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:00.767236 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:17:00.767461 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:00.767464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:00.767520 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:00.767618 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:00.768614 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.769231 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.770063 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:00.770289 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:00.770372 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:00.773264 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:00.773268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:00.773270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:00.773292 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:00.775032 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:00.775605 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:00.775608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:00.775610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:00.775713 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:00.776053 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:00.776057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:00.776119 (+ 62us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:00.776366 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:00.776471 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:00.776890 (+ 419us) server_negotiation.cc:299] Negotiation successful
0806 16:17:00.777015 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":150,"threads_started":1}
I20260806 16:17:00.783283 13977 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:33572:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 16:17:00.784880 13977 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:17:00.792258 14624 tablet_service.cc:1467] Processing CreateTablet for tablet bae2facaa2b14b1a9966c06898a913d9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=eeb2bd521059401781fdc6d8fdc35037]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:17:00.792201 14412 tablet_service.cc:1467] Processing CreateTablet for tablet bae2facaa2b14b1a9966c06898a913d9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=eeb2bd521059401781fdc6d8fdc35037]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:17:00.792201 14194 tablet_service.cc:1467] Processing CreateTablet for tablet bae2facaa2b14b1a9966c06898a913d9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=eeb2bd521059401781fdc6d8fdc35037]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:17:00.792811 14194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bae2facaa2b14b1a9966c06898a913d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:00.792812 14624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bae2facaa2b14b1a9966c06898a913d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:00.792812 14412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bae2facaa2b14b1a9966c06898a913d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:00.796370 14758 tablet_bootstrap.cc:492] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f: Bootstrap starting.
I20260806 16:17:00.796372 14759 tablet_bootstrap.cc:492] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b: Bootstrap starting.
I20260806 16:17:00.797003 14756 tablet_bootstrap.cc:492] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad: Bootstrap starting.
I20260806 16:17:00.797564 14758 tablet_bootstrap.cc:654] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:00.797582 14759 tablet_bootstrap.cc:654] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:00.798218 14756 tablet_bootstrap.cc:654] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:00.799105 14758 tablet_bootstrap.cc:492] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f: No bootstrap required, opened a new log
I20260806 16:17:00.799214 14758 ts_tablet_manager.cc:1397] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:00.799363 14759 tablet_bootstrap.cc:492] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b: No bootstrap required, opened a new log
I20260806 16:17:00.799463 14759 ts_tablet_manager.cc:1397] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:00.799779 14756 tablet_bootstrap.cc:492] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad: No bootstrap required, opened a new log
I20260806 16:17:00.799855 14756 ts_tablet_manager.cc:1397] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 16:17:00.799782 14758 raft_consensus.cc:348] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.799932 14758 raft_consensus.cc:374] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:00.799965 14758 raft_consensus.cc:729] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf8f17616a54f409d70f72f82b2a82f, State: Initialized, Role: FOLLOWER
I20260806 16:17:00.799891 14759 raft_consensus.cc:348] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.800010 14759 raft_consensus.cc:374] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:00.800042 14759 raft_consensus.cc:729] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a21f64bbe6184d0585356b0b6e0f571b, State: Initialized, Role: FOLLOWER
I20260806 16:17:00.800148 14758 consensus_queue.cc:260] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.800211 14759 consensus_queue.cc:260] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.800539 14759 ts_tablet_manager.cc:1428] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:17:00.800558 14768 tablet_replica.cc:439] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } } }
I20260806 16:17:00.800733 14768 tablet_replica.cc:442] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:00.801826 14756 raft_consensus.cc:348] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.801959 14756 raft_consensus.cc:374] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:00.801986 14756 raft_consensus.cc:729] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f06d38a4e09f4651a7c73ccd8676f4ad, State: Initialized, Role: FOLLOWER
I20260806 16:17:00.802105 14756 consensus_queue.cc:260] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:00.802451 14756 ts_tablet_manager.cc:1428] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 16:17:00.802381 14794 tablet_replica.cc:439] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } } }
I20260806 16:17:00.802604 14794 tablet_replica.cc:442] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:00.802862 14772 tablet_replica.cc:439] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } } }
I20260806 16:17:00.803046 14772 tablet_replica.cc:442] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:00.803391 14758 ts_tablet_manager.cc:1428] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260806 16:17:01.083546 14794 raft_consensus.cc:482] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:01.083781 14794 raft_consensus.cc:504] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:01.084527 14794 leader_election.cc:288] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf8f17616a54f409d70f72f82b2a82f (127.11.245.129:45401), f06d38a4e09f4651a7c73ccd8676f4ad (127.11.245.131:34311)
I20260806 16:17:01.085250 14222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bae2facaa2b14b1a9966c06898a913d9" candidate_uuid: "a21f64bbe6184d0585356b0b6e0f571b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf8f17616a54f409d70f72f82b2a82f" is_pre_election: true
I20260806 16:17:01.085274 14650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bae2facaa2b14b1a9966c06898a913d9" candidate_uuid: "a21f64bbe6184d0585356b0b6e0f571b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" is_pre_election: true
I20260806 16:17:01.085490 14222 raft_consensus.cc:2454] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a21f64bbe6184d0585356b0b6e0f571b in term 0.
I20260806 16:17:01.085511 14650 raft_consensus.cc:2454] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a21f64bbe6184d0585356b0b6e0f571b in term 0.
I20260806 16:17:01.085904 14333 leader_election.cc:302] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a21f64bbe6184d0585356b0b6e0f571b, f06d38a4e09f4651a7c73ccd8676f4ad; no voters:
I20260806 16:17:01.086148 14794 raft_consensus.cc:2781] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:01.086222 14794 raft_consensus.cc:482] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:01.086263 14794 raft_consensus.cc:3037] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:01.087646 14794 raft_consensus.cc:504] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:01.088107 14794 leader_election.cc:288] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [CANDIDATE]: Term 1 election: Requested vote from peers bdf8f17616a54f409d70f72f82b2a82f (127.11.245.129:45401), f06d38a4e09f4651a7c73ccd8676f4ad (127.11.245.131:34311)
I20260806 16:17:01.088565 14222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bae2facaa2b14b1a9966c06898a913d9" candidate_uuid: "a21f64bbe6184d0585356b0b6e0f571b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf8f17616a54f409d70f72f82b2a82f"
I20260806 16:17:01.088587 14650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bae2facaa2b14b1a9966c06898a913d9" candidate_uuid: "a21f64bbe6184d0585356b0b6e0f571b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad"
I20260806 16:17:01.088724 14222 raft_consensus.cc:3037] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:01.088737 14650 raft_consensus.cc:3037] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:01.089999 14650 raft_consensus.cc:2454] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a21f64bbe6184d0585356b0b6e0f571b in term 1.
I20260806 16:17:01.090080 14222 raft_consensus.cc:2454] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a21f64bbe6184d0585356b0b6e0f571b in term 1.
I20260806 16:17:01.090479 14333 leader_election.cc:302] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a21f64bbe6184d0585356b0b6e0f571b, f06d38a4e09f4651a7c73ccd8676f4ad; no voters:
I20260806 16:17:01.090732 14794 raft_consensus.cc:2781] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:01.090958 14794 raft_consensus.cc:686] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [term 1 LEADER]: Becoming Leader. State: Replica: a21f64bbe6184d0585356b0b6e0f571b, State: Running, Role: LEADER
I20260806 16:17:01.091156 14794 consensus_queue.cc:237] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } }
I20260806 16:17:01.092008 14827 tablet_replica.cc:439] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b: TxnStatusTablet state changed. Reason: New leader a21f64bbe6184d0585356b0b6e0f571b. Latest consensus state: current_term: 1 leader_uuid: "a21f64bbe6184d0585356b0b6e0f571b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } } }
I20260806 16:17:01.092159 14827 tablet_replica.cc:442] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:17:01.092530 14829 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 16:17:01.092794 14829 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 16:17:01.093712 13974 catalog_manager.cc:5495] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b reported cstate change: term changed from 0 to 1, leader changed from <none> to a21f64bbe6184d0585356b0b6e0f571b (127.11.245.130). New cstate: current_term: 1 leader_uuid: "a21f64bbe6184d0585356b0b6e0f571b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } health_report { overall_health: UNKNOWN } } }
I20260806 16:17:01.178125 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:01.173879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37500 (local address 127.11.245.130:41367)
0806 16:17:01.174199 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:01.174203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:01.174219 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:17:01.174413 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:01.174415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:01.174476 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:01.174590 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:01.174927 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.175528 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.176859 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.177118 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.177205 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:01.177590 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:01.177689 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:01.177741 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:01.177890 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 16:17:01.177953 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":133,"threads_started":1}
I20260806 16:17:01.178179 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:01.173752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:41367 (local address 127.0.0.1:37500)
0806 16:17:01.174100 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:01.174119 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:01.174262 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:01.174646 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:01.174650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:01.174659 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:17:01.174808 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:01.174815 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.175654 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.175658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:01.176723 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:01.176738 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.177246 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.177250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:01.177344 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:01.177423 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:17:01.177776 (+ 353us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:17:01.177794 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 16:17:01.177870 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 16:17:01.177986 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196}
I20260806 16:17:01.182487 14650 raft_consensus.cc:1261] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad [term 1 FOLLOWER]: Refusing update from remote peer a21f64bbe6184d0585356b0b6e0f571b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:01.182577 14222 raft_consensus.cc:1261] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f [term 1 FOLLOWER]: Refusing update from remote peer a21f64bbe6184d0585356b0b6e0f571b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:01.183043 14827 consensus_queue.cc:1035] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:01.183314 14794 consensus_queue.cc:1035] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:01.184876 14772 tablet_replica.cc:439] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad: TxnStatusTablet state changed. Reason: New leader a21f64bbe6184d0585356b0b6e0f571b. Latest consensus state: current_term: 1 leader_uuid: "a21f64bbe6184d0585356b0b6e0f571b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } } }
I20260806 16:17:01.185034 14772 tablet_replica.cc:442] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:01.184954 14768 tablet_replica.cc:439] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f: TxnStatusTablet state changed. Reason: New leader a21f64bbe6184d0585356b0b6e0f571b. Latest consensus state: current_term: 1 leader_uuid: "a21f64bbe6184d0585356b0b6e0f571b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } } }
I20260806 16:17:01.185110 14768 tablet_replica.cc:442] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:01.187642 14794 tablet_replica.cc:439] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a21f64bbe6184d0585356b0b6e0f571b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } } }
I20260806 16:17:01.187768 14794 tablet_replica.cc:442] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:17:01.188773 14827 tablet_replica.cc:439] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a21f64bbe6184d0585356b0b6e0f571b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } } }
I20260806 16:17:01.188921 14827 tablet_replica.cc:442] T bae2facaa2b14b1a9966c06898a913d9 P a21f64bbe6184d0585356b0b6e0f571b: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:17:01.189150 14768 tablet_replica.cc:439] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a21f64bbe6184d0585356b0b6e0f571b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } } }
I20260806 16:17:01.189275 14768 tablet_replica.cc:442] T bae2facaa2b14b1a9966c06898a913d9 P bdf8f17616a54f409d70f72f82b2a82f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:01.191763 14772 tablet_replica.cc:439] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a21f64bbe6184d0585356b0b6e0f571b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 } } peers { permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 } } peers { permanent_uuid: "f06d38a4e09f4651a7c73ccd8676f4ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 34311 } } }
I20260806 16:17:01.191942 14772 tablet_replica.cc:442] T bae2facaa2b14b1a9966c06898a913d9 P f06d38a4e09f4651a7c73ccd8676f4ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:01.201517 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:01.196856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58762 (local address 127.11.245.131:34311)
0806 16:17:01.197091 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:01.197095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:01.197174 (+ 79us) server_negotiation.cc:407] Connection header received
0806 16:17:01.197391 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:01.197393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:01.197456 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:01.197595 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:01.197917 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.198473 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.199685 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.199917 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.199993 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:01.200348 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:01.200997 (+ 649us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:01.201100 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:01.201209 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 16:17:01.201267 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":71,"threads_started":1}
I20260806 16:17:01.217336 14772 consensus_queue.cc:1035] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "a21f64bbe6184d0585356b0b6e0f571b" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:01.221575 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:01.212683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:44335 (local address 127.11.245.130:41367)
0806 16:17:01.212897 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:01.212903 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:01.216142 (+ 3239us) server_negotiation.cc:407] Connection header received
0806 16:17:01.216435 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:01.216440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:01.216512 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:01.216577 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:01.217128 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.218067 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.219745 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.220177 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.220274 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:01.220809 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:01.220937 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:01.220999 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:01.221201 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 16:17:01.221281 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260806 16:17:01.222366 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:01.212380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:41367 (local address 127.11.245.131:44335)
0806 16:17:01.215989 (+ 3609us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:01.216013 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:01.216265 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:01.216786 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:01.216790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:01.216801 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:17:01.216966 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:01.216974 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.218426 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.218430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:01.219576 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:01.219591 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.220428 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.220432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:01.220532 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:01.220608 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:17:01.221081 (+ 473us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:17:01.221088 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 16:17:01.222075 (+ 987us) client_negotiation.cc:240] Negotiation successful
0806 16:17:01.222192 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3430,"thread_start_us":170,"threads_started":1}
I20260806 16:17:01.227421 14772 consensus_queue.cc:1035] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf8f17616a54f409d70f72f82b2a82f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:01.260387 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:01.255126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:54815 (local address 127.11.245.131:34311)
0806 16:17:01.255272 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:01.255277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:01.255755 (+ 478us) server_negotiation.cc:407] Connection header received
0806 16:17:01.255947 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:01.255955 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:01.256012 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:01.256127 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:01.256458 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.257349 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.258613 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.259104 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.259229 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:01.259699 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:01.259821 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:01.259912 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:01.260112 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 16:17:01.260193 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 16:17:01.260429 14863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:01.255016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:34311 (local address 127.11.245.130:54815)
0806 16:17:01.255657 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:01.255675 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:01.255813 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:01.256143 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:01.256147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:01.256156 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:17:01.256317 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:01.256327 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.257542 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.257545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:01.258430 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:01.258466 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.259342 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.259347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:01.259459 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:01.259536 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:17:01.259945 (+ 409us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:17:01.259955 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 16:17:01.260052 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 16:17:01.260222 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":134,"threads_started":1}
I20260806 16:17:01.293226 13974 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:01.293486 13974 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:01.296945 13974 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3641097f8d954eaf9fbf075936b18b28: Sending DeleteTablet for 3 replicas of tablet b7b7fb70dc7746e59c53078145fa0685
I20260806 16:17:01.298075 14194 tablet_service.cc:1514] Processing DeleteTablet for tablet b7b7fb70dc7746e59c53078145fa0685 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:01 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:59188
I20260806 16:17:01.298636 14624 tablet_service.cc:1514] Processing DeleteTablet for tablet b7b7fb70dc7746e59c53078145fa0685 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:01 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:58748
I20260806 16:17:01.298687 14871 tablet_replica.cc:330] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f: stopping tablet replica
I20260806 16:17:01.299093 14871 raft_consensus.cc:2229] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:01.299196 14412 tablet_service.cc:1514] Processing DeleteTablet for tablet b7b7fb70dc7746e59c53078145fa0685 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:01 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:37488
I20260806 16:17:01.299438 14871 raft_consensus.cc:2258] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:01.300203 14873 tablet_replica.cc:330] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad: stopping tablet replica
I20260806 16:17:01.300384 14873 raft_consensus.cc:2229] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:17:01.300832 14871 ts_tablet_manager.cc:1905] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:01.300865 14873 raft_consensus.cc:2258] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:01.301311 14875 tablet_replica.cc:330] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b: stopping tablet replica
I20260806 16:17:01.301497 14875 raft_consensus.cc:2229] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:01.301646 14875 raft_consensus.cc:2258] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:01.302596 14875 ts_tablet_manager.cc:1905] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:01.304313 14873 ts_tablet_manager.cc:1905] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:01.306376 14871 ts_tablet_manager.cc:1918] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:17:01.306556 14871 log.cc:1196] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-0/wal/wals/b7b7fb70dc7746e59c53078145fa0685
I20260806 16:17:01.307039 14871 ts_tablet_manager.cc:1939] T b7b7fb70dc7746e59c53078145fa0685 P bdf8f17616a54f409d70f72f82b2a82f: Deleting consensus metadata
I20260806 16:17:01.308182 13956 catalog_manager.cc:4841] TS bdf8f17616a54f409d70f72f82b2a82f (127.11.245.129:45401): tablet b7b7fb70dc7746e59c53078145fa0685 (table test-table [id=60d16fa3ee034a0a9cd2d6462c265f66]) successfully deleted
Aug 06 16:17:01 dist-test-slave-t3q3 krb5kdc[13907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033019, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.245.129@KRBTEST.COM
I20260806 16:17:01.310894 14875 ts_tablet_manager.cc:1918] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:17:01.311017 14875 log.cc:1196] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/b7b7fb70dc7746e59c53078145fa0685
I20260806 16:17:01.311324 14875 ts_tablet_manager.cc:1939] T b7b7fb70dc7746e59c53078145fa0685 P a21f64bbe6184d0585356b0b6e0f571b: Deleting consensus metadata
I20260806 16:17:01.312495 13960 catalog_manager.cc:4841] TS a21f64bbe6184d0585356b0b6e0f571b (127.11.245.130:41367): tablet b7b7fb70dc7746e59c53078145fa0685 (table test-table [id=60d16fa3ee034a0a9cd2d6462c265f66]) successfully deleted
I20260806 16:17:01.313448 14873 ts_tablet_manager.cc:1918] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:17:01.313556 14873 log.cc:1196] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/b7b7fb70dc7746e59c53078145fa0685
I20260806 16:17:01.313895 14873 ts_tablet_manager.cc:1939] T b7b7fb70dc7746e59c53078145fa0685 P f06d38a4e09f4651a7c73ccd8676f4ad: Deleting consensus metadata
I20260806 16:17:01.314913 13956 catalog_manager.cc:4841] TS f06d38a4e09f4651a7c73ccd8676f4ad (127.11.245.131:34311): tablet b7b7fb70dc7746e59c53078145fa0685 (table test-table [id=60d16fa3ee034a0a9cd2d6462c265f66]) successfully deleted
I20260806 16:17:01.316493 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:01.299978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59194 (local address 127.11.245.129:45401)
0806 16:17:01.301642 (+ 1664us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:01.301650 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:01.302351 (+ 701us) server_negotiation.cc:407] Connection header received
0806 16:17:01.302416 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:01.302419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:01.302530 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:01.302638 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:01.303760 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.304628 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.305818 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.306209 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.306321 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:01.310855 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:01.310880 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:01.310904 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:01.310941 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:01.313969 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:01.314534 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:01.314542 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:01.314544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:01.314677 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:01.315060 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:01.315063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:01.315065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:01.315462 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:01.315672 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:01.315980 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 16:17:01.316209 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1313,"thread_start_us":149,"threads_started":1}
W20260806 16:17:01.317108 14288 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59194
I20260806 16:17:01.328977 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:01.319611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33574 (local address 127.11.245.190:43743)
0806 16:17:01.319932 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:01.319938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:01.319968 (+ 30us) server_negotiation.cc:407] Connection header received
0806 16:17:01.320025 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:01.320028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:01.320119 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:01.320217 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:01.321035 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.321729 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.322757 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:01.323051 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:01.323174 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:01.325032 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:01.325036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:01.325038 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:01.325066 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:01.326875 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:01.327372 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:01.327379 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:01.327383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:01.327479 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:01.327787 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:01.327791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:01.327793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:01.328042 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:01.328276 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:01.328553 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 16:17:01.328708 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":99,"threads_started":1}
W20260806 16:17:01.329591 13974 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33574
I20260806 16:17:01.331733 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 14086
I20260806 16:17:01.357959 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 14312
I20260806 16:17:01.386909 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 14530
I20260806 16:17:01.410446 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 13930
2026-08-06T16:17:01Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4256 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:17:01 dist-test-slave-t3q3 krb5kdc[14887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:17:01 dist-test-slave-t3q3 krb5kdc[14887](info): set up 2 sockets
Aug 06 16:17:01 dist-test-slave-t3q3 krb5kdc[14887](info): commencing operation
krb5kdc: starting...
W20260806 16:17:02.713060 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.001s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:17:02 dist-test-slave-t3q3 krb5kdc[14887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:17:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:17:02.876773 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:36821
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:36821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:03.021975 14907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:03.022436 14907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:03.022534 14907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:03.027089 14907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:03.027207 14907 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 16:17:03.027242 14907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:03.027288 14907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:03.027329 14907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:03.027366 14907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:17:03.033900 14907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:36821
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:36821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:03.035585 14907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:03.037075 14907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:03.048051 14916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:03.048139 14922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:03.048094 14918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:03.048566 14907 server_base.cc:1034] running on GCE node
I20260806 16:17:03.049371 14907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:03.050980 14907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:03.052390 14907 hybrid_clock.cc:648] HybridClock initialized: now 1786033023052302 us; error 102 us; skew 500 ppm
Aug 06 16:17:03 dist-test-slave-t3q3 krb5kdc[14887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033023, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:03.056037 14907 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:17:03.057647 14907 webserver.cc:460] Webserver started at http://127.11.245.190:42641/ using document root <none> and password file <none>
I20260806 16:17:03.058415 14907 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:03.058490 14907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:03.058735 14907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:03.061617 14907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "88a1e6b33c7f44b2bfacfaf186f52f0d"
format_stamp: "Formatted at 2026-08-06 16:17:03 on dist-test-slave-t3q3"
server_key: "bab2abfb3870549ae19aa54b78f96154"
server_key_iv: "a0774470a8d2c79940c6b1444e757807"
server_key_version: "encryptionkey@0"
I20260806 16:17:03.062314 14907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "88a1e6b33c7f44b2bfacfaf186f52f0d"
format_stamp: "Formatted at 2026-08-06 16:17:03 on dist-test-slave-t3q3"
server_key: "bab2abfb3870549ae19aa54b78f96154"
server_key_iv: "a0774470a8d2c79940c6b1444e757807"
server_key_version: "encryptionkey@0"
I20260806 16:17:03.067183 14907 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:17:03.070267 14932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:03.071928 14907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:03.072118 14907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "88a1e6b33c7f44b2bfacfaf186f52f0d"
format_stamp: "Formatted at 2026-08-06 16:17:03 on dist-test-slave-t3q3"
server_key: "bab2abfb3870549ae19aa54b78f96154"
server_key_iv: "a0774470a8d2c79940c6b1444e757807"
server_key_version: "encryptionkey@0"
I20260806 16:17:03.072261 14907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:03.099238 14907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:03.103957 14907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:03.104372 14907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:03.115545 14907 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:36821
I20260806 16:17:03.115568 15036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:36821 every 8 connection(s)
I20260806 16:17:03.116986 14907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:17:03.118058 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 14907
I20260806 16:17:03.118191 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:17:03.118557 12246 external_mini_cluster.cc:1428] Setting key 909881d1125a7eb0cbb08f6152d34b7e
I20260806 16:17:03.121676 15038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 16:17:03 dist-test-slave-t3q3 krb5kdc[14887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:03.131506 15038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d: Bootstrap starting.
I20260806 16:17:03.134912 15038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:03.136546 15038 log.cc:824] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:03.137706 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:03.120609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33008 (local address 127.11.245.190:36821)
0806 16:17:03.121130 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:03.121151 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:03.121199 (+ 48us) server_negotiation.cc:407] Connection header received
0806 16:17:03.122082 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:03.122106 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:03.122569 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:03.122948 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:03.124172 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.125348 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.126748 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.127286 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.127471 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:03.131254 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:03.131261 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:03.131282 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:03.131336 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:03.133503 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:03.134130 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:03.134137 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:03.134144 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:03.134287 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:03.134642 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:03.134646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:03.134649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:03.135269 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:03.135514 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:03.135880 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 16:17:03.136329 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":179,"threads_started":1}
I20260806 16:17:03.139624 15038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d: No bootstrap required, opened a new log
I20260806 16:17:03.144826 15038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88a1e6b33c7f44b2bfacfaf186f52f0d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36821 } }
I20260806 16:17:03.145299 15038 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:03.145371 15038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a1e6b33c7f44b2bfacfaf186f52f0d, State: Initialized, Role: FOLLOWER
I20260806 16:17:03.146027 15038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88a1e6b33c7f44b2bfacfaf186f52f0d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36821 } }
I20260806 16:17:03.146276 15038 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:17:03.146389 15038 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:17:03.146538 15038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:03.147908 15038 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88a1e6b33c7f44b2bfacfaf186f52f0d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36821 } }
I20260806 16:17:03.148403 15038 leader_election.cc:302] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88a1e6b33c7f44b2bfacfaf186f52f0d; no voters:
I20260806 16:17:03.148838 15038 leader_election.cc:288] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:17:03.149001 15051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:03.149288 15051 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [term 1 LEADER]: Becoming Leader. State: Replica: 88a1e6b33c7f44b2bfacfaf186f52f0d, State: Running, Role: LEADER
I20260806 16:17:03.149732 15051 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88a1e6b33c7f44b2bfacfaf186f52f0d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36821 } }
I20260806 16:17:03.150566 15038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:03.152050 15052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88a1e6b33c7f44b2bfacfaf186f52f0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88a1e6b33c7f44b2bfacfaf186f52f0d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36821 } } }
I20260806 16:17:03.152261 15052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:03.152613 15053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88a1e6b33c7f44b2bfacfaf186f52f0d. Latest consensus state: current_term: 1 leader_uuid: "88a1e6b33c7f44b2bfacfaf186f52f0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88a1e6b33c7f44b2bfacfaf186f52f0d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36821 } } }
I20260806 16:17:03.152710 15053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:03.152761 15062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:03.157135 15062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:03.164858 15062 catalog_manager.cc:1347] Generated new cluster ID: 0a2424bce48c486d99f61478affbdf4c
I20260806 16:17:03.165009 15062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:03.178047 15062 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:17:03.179747 15062 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:03.189237 15062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d: Generated new TSK 0
I20260806 16:17:03.190481 15062 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:17:03.232409 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:36821
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:03.380016 15077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:03.380419 15077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:03.380476 15077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:03.384793 15077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:03.384876 15077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:03.384922 15077 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 16:17:03.384938 15077 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:17:03.385016 15077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:17:03.391245 15077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.245.190:36821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:03.392931 15077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:03.394378 15077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:03.407277 15097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:03.407305 15092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:03.407455 15077 server_base.cc:1034] running on GCE node
W20260806 16:17:03.407790 15093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:03.408396 15077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:03.409334 15077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:03.410544 15077 hybrid_clock.cc:648] HybridClock initialized: now 1786033023410510 us; error 40 us; skew 500 ppm
Aug 06 16:17:03 dist-test-slave-t3q3 krb5kdc[14887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033023, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:03.414227 15077 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:17:03.415405 15077 webserver.cc:460] Webserver started at http://127.11.245.129:35281/ using document root <none> and password file <none>
I20260806 16:17:03.416203 15077 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:03.416281 15077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:03.416517 15077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:03.419173 15077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "945ddb389c3f48b6ab135573c825de28"
format_stamp: "Formatted at 2026-08-06 16:17:03 on dist-test-slave-t3q3"
server_key: "8c507767fb21e87f918a36d75469c3ca"
server_key_iv: "d40787d9ce2ec48c1551b41cd6d29a08"
server_key_version: "encryptionkey@0"
I20260806 16:17:03.419870 15077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "945ddb389c3f48b6ab135573c825de28"
format_stamp: "Formatted at 2026-08-06 16:17:03 on dist-test-slave-t3q3"
server_key: "8c507767fb21e87f918a36d75469c3ca"
server_key_iv: "d40787d9ce2ec48c1551b41cd6d29a08"
server_key_version: "encryptionkey@0"
I20260806 16:17:03.425066 15077 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 16:17:03.428607 15106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:03.430472 15077 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260806 16:17:03.430635 15077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "945ddb389c3f48b6ab135573c825de28"
format_stamp: "Formatted at 2026-08-06 16:17:03 on dist-test-slave-t3q3"
server_key: "8c507767fb21e87f918a36d75469c3ca"
server_key_iv: "d40787d9ce2ec48c1551b41cd6d29a08"
server_key_version: "encryptionkey@0"
I20260806 16:17:03.430881 15077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:03.451321 15077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:03.455945 15077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:03.456246 15077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:03.458316 15077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:03.458393 15077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:03.458489 15077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:03.458530 15077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:03.472497 15077 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:39757
Aug 06 16:17:03 dist-test-slave-t3q3 krb5kdc[14887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033023, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:03.472514 15270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:39757 every 8 connection(s)
I20260806 16:17:03.473881 15077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:17:03.478897 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:03.459607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:35959 (local address 127.11.245.190:36821)
0806 16:17:03.459812 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:03.459820 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:03.461082 (+ 1262us) server_negotiation.cc:407] Connection header received
0806 16:17:03.462836 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:03.462843 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:03.462935 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:03.463046 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:03.465481 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.466572 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.468846 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.469155 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.469321 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:03.473590 (+ 4269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:03.473594 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:03.473597 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:03.473633 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:03.475779 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:03.476505 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:03.476510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:03.476512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:03.476614 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:03.477279 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:03.477282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:03.477284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:03.477560 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:03.477782 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:03.478496 (+ 714us) server_negotiation.cc:299] Negotiation successful
0806 16:17:03.478688 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 16:17:03.480643 15125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:03.460219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:36821 (local address 127.11.245.129:35959)
0806 16:17:03.460855 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:03.460928 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:03.462460 (+ 1532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:03.463352 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:03.463362 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:03.464109 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:03.465114 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:03.465148 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.466770 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.466777 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:03.468649 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:03.468667 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.469393 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.469402 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:03.469715 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:03.470747 (+ 1032us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:03.470780 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:03.473272 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:03.475959 (+ 2687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:03.475967 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:03.475987 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:03.476376 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:03.476894 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:03.476900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:03.476909 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:03.477149 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:03.477786 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:03.477797 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:03.478183 (+ 386us) client_negotiation.cc:769] Sending connection context
0806 16:17:03.478422 (+ 239us) client_negotiation.cc:240] Negotiation successful
0806 16:17:03.478870 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":136,"threads_started":1}
I20260806 16:17:03.482622 15271 heartbeater.cc:347] Connected to a master server at 127.11.245.190:36821
I20260806 16:17:03.482874 15271 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:03.483693 15271 heartbeater.cc:511] Master 127.11.245.190:36821 requested a full tablet report, sending...
I20260806 16:17:03.484292 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 15077
I20260806 16:17:03.484479 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:17:03.484879 12246 external_mini_cluster.cc:1428] Setting key a67a5d4dd10bc255bba01cfd7e43e9e0
I20260806 16:17:03.486876 14959 ts_manager.cc:194] Registered new tserver with Master: 945ddb389c3f48b6ab135573c825de28 (127.11.245.129:39757)
I20260806 16:17:03.488880 14959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:35959
I20260806 16:17:03.494887 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:03.485953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:50925 (local address 127.11.245.190:36821)
0806 16:17:03.486383 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:03.486388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:03.486405 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:17:03.486557 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:03.486560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:03.486613 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:03.486757 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:03.487414 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.488012 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.488953 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.489288 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.489404 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:03.491030 (+ 1626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:03.491036 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:03.491041 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:03.491080 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:03.492943 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:03.493488 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:03.493492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:03.493494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:03.493591 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:03.493904 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:03.493910 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:03.493915 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:03.494132 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:03.494255 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:03.494539 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 16:17:03.494679 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345}
I20260806 16:17:03.494912 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:03.485896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:36821 (local address 127.11.245.129:50925)
0806 16:17:03.486303 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:03.486322 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:03.486436 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:03.486832 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:03.486838 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:03.487139 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:03.487281 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:03.487289 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.488186 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.488191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:03.488815 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:03.488823 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.489425 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.489429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:03.489536 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:03.490287 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:03.490301 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:03.490844 (+ 543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:03.493122 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:03.493126 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:03.493128 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:03.493370 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:03.493694 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:03.493698 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:03.493700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:03.493804 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:03.494241 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:03.494245 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:03.494355 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 16:17:03.494475 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 16:17:03.494661 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":148,"threads_started":1}
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:17:03.524652 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:36821
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:03.681034 15281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:03.681561 15281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:03.681715 15281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:03.686908 15281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:03.687004 15281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:03.687052 15281 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 16:17:03.687067 15281 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:17:03.687103 15281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:17:03.694094 15281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.245.190:36821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:03.695724 15281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:03.697167 15281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:03.711184 15289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:03.711177 15293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:03.711297 15281 server_base.cc:1034] running on GCE node
W20260806 16:17:03.711177 15290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:03.712297 15281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:03.713344 15281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:03.714589 15281 hybrid_clock.cc:648] HybridClock initialized: now 1786033023714551 us; error 56 us; skew 500 ppm
Aug 06 16:17:03 dist-test-slave-t3q3 krb5kdc[14887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033023, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:03.718883 15281 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:17:03.720399 15281 webserver.cc:460] Webserver started at http://127.11.245.130:38157/ using document root <none> and password file <none>
I20260806 16:17:03.721396 15281 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:03.721521 15281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:03.721869 15281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:03.725001 15281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "12e54dd8327a4d48a27207bc73dbfd08"
format_stamp: "Formatted at 2026-08-06 16:17:03 on dist-test-slave-t3q3"
server_key: "3f9e8b5fb55f18a569285c0f457892ea"
server_key_iv: "c51d184f7ee64aeb6840ef3d8d35f1cc"
server_key_version: "encryptionkey@0"
I20260806 16:17:03.726131 15281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "12e54dd8327a4d48a27207bc73dbfd08"
format_stamp: "Formatted at 2026-08-06 16:17:03 on dist-test-slave-t3q3"
server_key: "3f9e8b5fb55f18a569285c0f457892ea"
server_key_iv: "c51d184f7ee64aeb6840ef3d8d35f1cc"
server_key_version: "encryptionkey@0"
I20260806 16:17:03.732242 15281 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:17:03.735585 15301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:03.737406 15281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:17:03.737576 15281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "12e54dd8327a4d48a27207bc73dbfd08"
format_stamp: "Formatted at 2026-08-06 16:17:03 on dist-test-slave-t3q3"
server_key: "3f9e8b5fb55f18a569285c0f457892ea"
server_key_iv: "c51d184f7ee64aeb6840ef3d8d35f1cc"
server_key_version: "encryptionkey@0"
I20260806 16:17:03.737736 15281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:03.763149 15281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:03.767447 15281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:03.767668 15281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:03.769699 15281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:03.769805 15281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:03.769935 15281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:03.770001 15281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 16:17:03 dist-test-slave-t3q3 krb5kdc[14887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033023, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:03.783427 15281 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:44595
I20260806 16:17:03.783478 15471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:44595 every 8 connection(s)
I20260806 16:17:03.784840 15281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:17:03.786578 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 15281
I20260806 16:17:03.786728 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:17:03.787130 12246 external_mini_cluster.cc:1428] Setting key 15b4a1759f75328f430276256f52b8c0
I20260806 16:17:03.788311 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:03.770732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:35321 (local address 127.11.245.190:36821)
0806 16:17:03.770946 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:03.770953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:03.772562 (+ 1609us) server_negotiation.cc:407] Connection header received
0806 16:17:03.774089 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:03.774097 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:03.774190 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:03.774325 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:03.776278 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.777456 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.779599 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.780025 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.780233 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:03.783690 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:03.783694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:03.783698 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:03.783739 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:03.785521 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:03.786178 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:03.786182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:03.786184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:03.786316 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:03.786743 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:03.786747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:03.786750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:03.787042 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:03.787159 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:03.787901 (+ 742us) server_negotiation.cc:299] Negotiation successful
0806 16:17:03.788103 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 16:17:03.789490 15316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:03.771751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:36821 (local address 127.11.245.130:35321)
0806 16:17:03.772346 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:03.772407 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:03.773751 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:03.774543 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:03.774552 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:03.775131 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:03.776019 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:03.776040 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.777622 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.777625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:03.779410 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:03.779424 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.780281 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.780285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:03.780470 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:03.781321 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:03.781347 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:03.783382 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:03.785685 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:03.785691 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:03.785709 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:03.786087 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:03.786453 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:03.786456 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:03.786457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:03.786608 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:03.787213 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:03.787223 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:03.787558 (+ 335us) client_negotiation.cc:769] Sending connection context
0806 16:17:03.787812 (+ 254us) client_negotiation.cc:240] Negotiation successful
0806 16:17:03.788255 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":174,"threads_started":1}
I20260806 16:17:03.791054 15472 heartbeater.cc:347] Connected to a master server at 127.11.245.190:36821
I20260806 16:17:03.791261 15472 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:03.792042 15472 heartbeater.cc:511] Master 127.11.245.190:36821 requested a full tablet report, sending...
I20260806 16:17:03.793668 14959 ts_manager.cc:194] Registered new tserver with Master: 12e54dd8327a4d48a27207bc73dbfd08 (127.11.245.130:44595)
I20260806 16:17:03.794519 14959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:35321
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
I20260806 16:17:03.806473 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:03.794071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:36821 (local address 127.11.245.130:49199)
0806 16:17:03.796362 (+ 2291us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:03.796383 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:03.796536 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:03.797038 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:03.797042 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:03.797539 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:03.797754 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:03.797770 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.798716 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.798723 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:03.799829 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:03.799846 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.800475 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.800479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:03.800615 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:03.801320 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:03.801343 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:03.801908 (+ 565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:03.804256 (+ 2348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:03.804260 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:03.804262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:03.804513 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:03.804999 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:03.805003 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:03.805007 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:03.805107 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:03.805745 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:03.805751 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:03.805907 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 16:17:03.806051 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 16:17:03.806253 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2145,"thread_start_us":129,"threads_started":1}
I20260806 16:17:03.806671 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:03.794142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:49199 (local address 127.11.245.190:36821)
0806 16:17:03.794274 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:03.794278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:03.796504 (+ 2226us) server_negotiation.cc:407] Connection header received
0806 16:17:03.796687 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:03.796693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:03.796771 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:03.796839 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:03.797946 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.798526 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.799991 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:03.800333 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:03.800433 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:03.802100 (+ 1667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:03.802103 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:03.802106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:03.802133 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:03.804005 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:03.804640 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:03.804645 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:03.804647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:03.804791 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:03.805227 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:03.805231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:03.805233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:03.805596 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:03.805744 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:03.806195 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 16:17:03.806413 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:17:03.828475 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:36821
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:04.011587 15487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:04.011986 15487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:04.012120 15487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:04.016995 15487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:04.017145 15487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:04.017239 15487 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 16:17:04.017290 15487 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:17:04.017386 15487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:17:04.025911 15487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.245.190:36821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:04.027573 15487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:04.029104 15487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:04.043076 15498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:04.043072 15496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:04.043062 15495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:04.043928 15487 server_base.cc:1034] running on GCE node
I20260806 16:17:04.044680 15487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:04.045620 15487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:04.049029 15487 hybrid_clock.cc:648] HybridClock initialized: now 1786033024048985 us; error 56 us; skew 500 ppm
Aug 06 16:17:04 dist-test-slave-t3q3 krb5kdc[14887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033024, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:04.053700 15487 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:17:04.055156 15487 webserver.cc:460] Webserver started at http://127.11.245.131:39347/ using document root <none> and password file <none>
I20260806 16:17:04.056017 15487 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:04.056190 15487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:04.056545 15487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:04.059576 15487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "349bce4fe2e7407bb3ad0407b5a8e236"
format_stamp: "Formatted at 2026-08-06 16:17:04 on dist-test-slave-t3q3"
server_key: "8281a7ccfba50eb1aaf6c73e1a20cee3"
server_key_iv: "a57053fdf5d042fffda9f4a65338045c"
server_key_version: "encryptionkey@0"
I20260806 16:17:04.060463 15487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "349bce4fe2e7407bb3ad0407b5a8e236"
format_stamp: "Formatted at 2026-08-06 16:17:04 on dist-test-slave-t3q3"
server_key: "8281a7ccfba50eb1aaf6c73e1a20cee3"
server_key_iv: "a57053fdf5d042fffda9f4a65338045c"
server_key_version: "encryptionkey@0"
I20260806 16:17:04.066216 15487 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:17:04.070400 15510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:04.072718 15487 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:17:04.072938 15487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "349bce4fe2e7407bb3ad0407b5a8e236"
format_stamp: "Formatted at 2026-08-06 16:17:04 on dist-test-slave-t3q3"
server_key: "8281a7ccfba50eb1aaf6c73e1a20cee3"
server_key_iv: "a57053fdf5d042fffda9f4a65338045c"
server_key_version: "encryptionkey@0"
I20260806 16:17:04.073145 15487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:04.091303 15487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:04.096004 15487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:04.096347 15487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:04.098677 15487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:04.098758 15487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:04.098841 15487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:04.098877 15487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:04.114549 15487 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:35401
Aug 06 16:17:04 dist-test-slave-t3q3 krb5kdc[14887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033024, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:04.116711 15487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:17:04.119791 15682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:35401 every 8 connection(s)
I20260806 16:17:04.121922 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 15487
I20260806 16:17:04.122076 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:17:04.122673 12246 external_mini_cluster.cc:1428] Setting key a8ab8de6d18f249b80dced14300ae4c9
I20260806 16:17:04.123113 15524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.100675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:36821 (local address 127.11.245.131:44041)
0806 16:17:04.101553 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:04.101627 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:04.103415 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:04.104963 (+ 1548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:04.104976 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:04.105719 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:04.106896 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.106933 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.108644 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.108652 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.110624 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.110640 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.111698 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.111707 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.112212 (+ 505us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.113377 (+ 1165us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:04.113399 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:04.115773 (+ 2374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:04.118617 (+ 2844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:04.118627 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:04.118654 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:04.119146 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:04.119622 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:04.119626 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:04.119630 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:04.120217 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:04.120862 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:04.120869 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:04.121240 (+ 371us) client_negotiation.cc:769] Sending connection context
0806 16:17:04.121539 (+ 299us) client_negotiation.cc:240] Negotiation successful
0806 16:17:04.121933 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":174,"threads_started":1}
I20260806 16:17:04.124552 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.099961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:44041 (local address 127.11.245.190:36821)
0806 16:17:04.100181 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:04.100190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:04.101780 (+ 1590us) server_negotiation.cc:407] Connection header received
0806 16:17:04.103795 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:04.103804 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:04.103911 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:04.103983 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:04.107213 (+ 3230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.108353 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.110864 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.111407 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.111612 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.116116 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:04.116123 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:04.116126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:04.116177 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:04.118416 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:04.119324 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:04.119331 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:04.119335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:04.119476 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:04.120370 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:04.120374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:04.120376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:04.120683 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:04.120771 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:04.124143 (+ 3372us) server_negotiation.cc:299] Negotiation successful
0806 16:17:04.124354 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 16:17:04.126114 15683 heartbeater.cc:347] Connected to a master server at 127.11.245.190:36821
I20260806 16:17:04.126367 15683 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:04.127231 15683 heartbeater.cc:511] Master 127.11.245.190:36821 requested a full tablet report, sending...
I20260806 16:17:04.128546 14960 ts_manager.cc:194] Registered new tserver with Master: 349bce4fe2e7407bb3ad0407b5a8e236 (127.11.245.131:35401)
I20260806 16:17:04.129516 14960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:44041
I20260806 16:17:04.140246 15690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.128965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:36821 (local address 127.11.245.131:41837)
0806 16:17:04.129360 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:04.129375 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:04.129501 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:04.129906 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:04.129911 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:04.130195 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:04.130478 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.130494 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.131641 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.131645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.132499 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.132511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.133102 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.133106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.133189 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.133888 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:04.133904 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:04.134376 (+ 472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:04.136699 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:04.136702 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:04.136704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:04.137314 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:04.137997 (+ 683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:04.138037 (+ 40us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:04.138042 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:04.138200 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:04.139486 (+ 1286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:04.139489 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:04.139611 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 16:17:04.139723 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 16:17:04.139936 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":126,"threads_started":1}
I20260806 16:17:04.140465 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:17:04.140525 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.129051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:41837 (local address 127.11.245.190:36821)
0806 16:17:04.129178 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:04.129183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:04.129473 (+ 290us) server_negotiation.cc:407] Connection header received
0806 16:17:04.129606 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:04.129609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:04.129663 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:04.129748 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:04.130629 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.131484 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.132621 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.132939 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.133068 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.134575 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:04.134579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:04.134582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:04.134611 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:04.136542 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:04.137606 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:04.137616 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:04.137618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:04.137732 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:04.138615 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:04.138622 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:04.138624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:04.139224 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:04.139386 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:04.139886 (+ 500us) server_negotiation.cc:299] Negotiation successful
0806 16:17:04.140288 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 16:17:04 dist-test-slave-t3q3 krb5kdc[14887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 16:17:04 dist-test-slave-t3q3 krb5kdc[14887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:04.177538 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.166970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33012 (local address 127.11.245.190:36821)
0806 16:17:04.167129 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:04.167134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:04.167278 (+ 144us) server_negotiation.cc:407] Connection header received
0806 16:17:04.167423 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:04.167426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:04.167492 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:04.167567 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:04.168588 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.169234 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.170145 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.170407 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.170528 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.173904 (+ 3376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:04.173909 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:04.173911 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:04.173947 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:04.175558 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:04.176132 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:04.176140 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:04.176144 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:04.176260 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:04.176580 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:04.176583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:04.176584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:04.176818 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:04.176941 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:04.177184 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 16:17:04.177361 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 16:17:04.181747 14960 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:17:04.186162 14960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:17:04.205302 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.199218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41322 (local address 127.11.245.130:44595)
0806 16:17:04.199751 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:04.199763 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:04.199809 (+ 46us) server_negotiation.cc:407] Connection header received
0806 16:17:04.199915 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:04.199922 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:04.200212 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:04.200417 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:04.200845 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.202121 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.203495 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.204448 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.204594 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.204765 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:04.204985 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 16:17:04.205108 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":145,"threads_started":1}
I20260806 16:17:04.205302 15708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.198230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:35401 (local address 127.0.0.1:54244)
0806 16:17:04.198728 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:04.198797 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:04.198964 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:04.200707 (+ 1743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:04.200710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:04.200724 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:04.200800 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.200805 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.202266 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.202270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.203605 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.203614 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.204715 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.204718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.204790 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.204806 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 16:17:04.204903 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 16:17:04.205071 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":64,"thread_start_us":144,"threads_started":1}
I20260806 16:17:04.205341 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.198660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:44595 (local address 127.0.0.1:41322)
0806 16:17:04.199432 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:04.199446 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:04.199543 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:04.200407 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:04.200414 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:04.200442 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:04.200678 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.200688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.202266 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.202270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.203349 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.203360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.204594 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.204599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.204743 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.204797 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 16:17:04.204945 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 16:17:04.205203 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":275,"thread_start_us":131,"threads_started":1}
I20260806 16:17:04.205837 15711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.199138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:39757 (local address 127.0.0.1:59132)
0806 16:17:04.199746 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:04.199764 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:04.199865 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:04.200627 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:04.200631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:04.200649 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:04.200792 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.200798 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.201943 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.201947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.203504 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.203516 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.204840 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.204844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.204961 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.204986 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 16:17:04.205059 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 16:17:04.205128 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":22,"thread_start_us":114,"threads_started":1}
I20260806 16:17:04.206174 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.198487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54244 (local address 127.11.245.131:35401)
0806 16:17:04.200207 (+ 1720us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:04.200218 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:04.200241 (+ 23us) server_negotiation.cc:407] Connection header received
0806 16:17:04.200348 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:04.200355 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:04.200554 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:04.200729 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:04.200903 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.202121 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.203738 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.204574 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.205700 (+ 1126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.205864 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:04.205925 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 16:17:04.206038 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1548,"thread_start_us":198,"threads_started":1}
I20260806 16:17:04.206271 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.199655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59132 (local address 127.11.245.129:39757)
0806 16:17:04.200010 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:04.200018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:04.200042 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:17:04.200152 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:04.200156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:04.200340 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:04.200475 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:04.200938 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.201799 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.203646 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.204473 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.204595 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.204765 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:04.206008 (+ 1243us) server_negotiation.cc:299] Negotiation successful
0806 16:17:04.206128 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":107,"threads_started":1}
I20260806 16:17:04.209508 15181 tablet_service.cc:1467] Processing CreateTablet for tablet 754da61692b34f00951dc49998a1141e (DEFAULT_TABLE table=test-table [id=bd74152e80054e50ba4be5985571f8fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:04.209530 15372 tablet_service.cc:1467] Processing CreateTablet for tablet 754da61692b34f00951dc49998a1141e (DEFAULT_TABLE table=test-table [id=bd74152e80054e50ba4be5985571f8fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:04.209530 15559 tablet_service.cc:1467] Processing CreateTablet for tablet 754da61692b34f00951dc49998a1141e (DEFAULT_TABLE table=test-table [id=bd74152e80054e50ba4be5985571f8fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:04.210920 15181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 754da61692b34f00951dc49998a1141e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:04.210987 15559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 754da61692b34f00951dc49998a1141e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:04.211470 15372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 754da61692b34f00951dc49998a1141e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:04.220716 15722 tablet_bootstrap.cc:492] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236: Bootstrap starting.
I20260806 16:17:04.222306 15725 tablet_bootstrap.cc:492] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28: Bootstrap starting.
I20260806 16:17:04.222711 15727 tablet_bootstrap.cc:492] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08: Bootstrap starting.
I20260806 16:17:04.225057 15727 tablet_bootstrap.cc:654] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:04.225060 15725 tablet_bootstrap.cc:654] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:04.225060 15722 tablet_bootstrap.cc:654] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:04.226176 15725 log.cc:824] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:04.226171 15722 log.cc:824] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:04.226217 15727 log.cc:824] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:04.229130 15727 tablet_bootstrap.cc:492] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08: No bootstrap required, opened a new log
I20260806 16:17:04.229140 15725 tablet_bootstrap.cc:492] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28: No bootstrap required, opened a new log
I20260806 16:17:04.229146 15722 tablet_bootstrap.cc:492] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236: No bootstrap required, opened a new log
I20260806 16:17:04.229463 15725 ts_tablet_manager.cc:1397] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 16:17:04.229463 15722 ts_tablet_manager.cc:1397] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 16:17:04.230010 15727 ts_tablet_manager.cc:1397] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 16:17:04.234352 15727 raft_consensus.cc:348] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } }
I20260806 16:17:04.234797 15727 raft_consensus.cc:374] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:04.234860 15727 raft_consensus.cc:729] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e54dd8327a4d48a27207bc73dbfd08, State: Initialized, Role: FOLLOWER
I20260806 16:17:04.235154 15722 raft_consensus.cc:348] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } }
I20260806 16:17:04.235525 15722 raft_consensus.cc:374] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:04.235600 15722 raft_consensus.cc:729] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349bce4fe2e7407bb3ad0407b5a8e236, State: Initialized, Role: FOLLOWER
I20260806 16:17:04.235560 15727 consensus_queue.cc:260] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } }
I20260806 16:17:04.235611 15725 raft_consensus.cc:348] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } }
I20260806 16:17:04.235932 15725 raft_consensus.cc:374] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:04.236006 15725 raft_consensus.cc:729] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945ddb389c3f48b6ab135573c825de28, State: Initialized, Role: FOLLOWER
I20260806 16:17:04.236167 15722 consensus_queue.cc:260] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } }
I20260806 16:17:04.236761 15472 heartbeater.cc:503] Master 127.11.245.190:36821 was elected leader, sending a full tablet report...
I20260806 16:17:04.237224 15727 ts_tablet_manager.cc:1428] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 16:17:04.237435 15722 ts_tablet_manager.cc:1428] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260806 16:17:04.237913 15683 heartbeater.cc:503] Master 127.11.245.190:36821 was elected leader, sending a full tablet report...
I20260806 16:17:04.237944 15725 consensus_queue.cc:260] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } }
I20260806 16:17:04.239341 15725 ts_tablet_manager.cc:1428] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260806 16:17:04.239820 15271 heartbeater.cc:503] Master 127.11.245.190:36821 was elected leader, sending a full tablet report...
W20260806 16:17:04.286653 15473 tablet.cc:2369] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:17:04.371910 15684 tablet.cc:2369] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:17:04.382863 15734 raft_consensus.cc:482] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:04.383275 15734 raft_consensus.cc:504] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } }
I20260806 16:17:04.385039 15734 leader_election.cc:288] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 349bce4fe2e7407bb3ad0407b5a8e236 (127.11.245.131:35401), 12e54dd8327a4d48a27207bc73dbfd08 (127.11.245.130:44595)
I20260806 16:17:04.389736 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.385567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:55457 (local address 127.11.245.131:35401)
0806 16:17:04.385748 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:04.385754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:04.386195 (+ 441us) server_negotiation.cc:407] Connection header received
0806 16:17:04.386400 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:04.386404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:04.386468 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:04.386569 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:04.386908 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.387491 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.388663 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.389106 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.389257 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.389364 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:04.389518 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 16:17:04.389598 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 16:17:04.389766 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.385411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:44595 (local address 127.11.245.129:50003)
0806 16:17:04.385917 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:04.385940 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:04.386108 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:04.386428 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:04.386431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:04.386449 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:04.386604 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.386611 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.387450 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.387454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.388480 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.388491 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.389196 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.389200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.389310 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.389348 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 16:17:04.389463 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 16:17:04.389555 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":65,"thread_start_us":168,"threads_started":1}
I20260806 16:17:04.389853 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.385673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:50003 (local address 127.11.245.130:44595)
0806 16:17:04.385829 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:04.385846 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:04.386039 (+ 193us) server_negotiation.cc:407] Connection header received
0806 16:17:04.386260 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:04.386263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:04.386316 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:04.386411 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:04.386727 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.387319 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.388619 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.389058 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.389199 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.389273 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:04.389582 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 16:17:04.389692 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 16:17:04.390281 15735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.385446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:35401 (local address 127.11.245.129:55457)
0806 16:17:04.386042 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:04.386056 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:04.386240 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:04.386582 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:04.386585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:04.386604 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:04.386747 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.386754 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.387620 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.387626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.388520 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.388531 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.389243 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.389247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.389361 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.389397 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 16:17:04.389491 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 16:17:04.389579 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":162,"threads_started":1}
I20260806 16:17:04.390798 15394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "754da61692b34f00951dc49998a1141e" candidate_uuid: "945ddb389c3f48b6ab135573c825de28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12e54dd8327a4d48a27207bc73dbfd08" is_pre_election: true
I20260806 16:17:04.391085 15587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "754da61692b34f00951dc49998a1141e" candidate_uuid: "945ddb389c3f48b6ab135573c825de28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" is_pre_election: true
I20260806 16:17:04.391268 15394 raft_consensus.cc:2454] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 945ddb389c3f48b6ab135573c825de28 in term 0.
I20260806 16:17:04.391506 15587 raft_consensus.cc:2454] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 945ddb389c3f48b6ab135573c825de28 in term 0.
I20260806 16:17:04.391968 15110 leader_election.cc:302] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12e54dd8327a4d48a27207bc73dbfd08, 945ddb389c3f48b6ab135573c825de28; no voters:
I20260806 16:17:04.392376 15734 raft_consensus.cc:2781] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:04.392485 15734 raft_consensus.cc:482] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:04.392534 15734 raft_consensus.cc:3037] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:04.394129 15734 raft_consensus.cc:504] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } }
I20260806 16:17:04.394780 15734 leader_election.cc:288] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [CANDIDATE]: Term 1 election: Requested vote from peers 349bce4fe2e7407bb3ad0407b5a8e236 (127.11.245.131:35401), 12e54dd8327a4d48a27207bc73dbfd08 (127.11.245.130:44595)
I20260806 16:17:04.395077 15394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "754da61692b34f00951dc49998a1141e" candidate_uuid: "945ddb389c3f48b6ab135573c825de28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12e54dd8327a4d48a27207bc73dbfd08"
I20260806 16:17:04.395076 15587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "754da61692b34f00951dc49998a1141e" candidate_uuid: "945ddb389c3f48b6ab135573c825de28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349bce4fe2e7407bb3ad0407b5a8e236"
I20260806 16:17:04.395241 15394 raft_consensus.cc:3037] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:04.395241 15587 raft_consensus.cc:3037] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:04.397022 15394 raft_consensus.cc:2454] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 945ddb389c3f48b6ab135573c825de28 in term 1.
I20260806 16:17:04.397447 15110 leader_election.cc:302] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12e54dd8327a4d48a27207bc73dbfd08, 945ddb389c3f48b6ab135573c825de28; no voters:
I20260806 16:17:04.397502 15587 raft_consensus.cc:2454] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 945ddb389c3f48b6ab135573c825de28 in term 1.
I20260806 16:17:04.397706 15734 raft_consensus.cc:2781] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:04.398068 15734 raft_consensus.cc:686] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [term 1 LEADER]: Becoming Leader. State: Replica: 945ddb389c3f48b6ab135573c825de28, State: Running, Role: LEADER
I20260806 16:17:04.398595 15734 consensus_queue.cc:237] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } }
I20260806 16:17:04.403163 14958 catalog_manager.cc:5495] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 945ddb389c3f48b6ab135573c825de28 (127.11.245.129). New cstate: current_term: 1 leader_uuid: "945ddb389c3f48b6ab135573c825de28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } health_report { overall_health: UNKNOWN } } }
Aug 06 16:17:04 dist-test-slave-t3q3 krb5kdc[14887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033023, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:04.472848 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.462867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33014 (local address 127.11.245.190:36821)
0806 16:17:04.463037 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:04.463042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:04.463290 (+ 248us) server_negotiation.cc:407] Connection header received
0806 16:17:04.463389 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:04.463391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:04.463521 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:04.463619 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:04.464668 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.465392 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.466267 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.466508 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.466598 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.469278 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:04.469282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:04.469284 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:04.469307 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:04.470832 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:04.471319 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:04.471322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:04.471324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:04.471403 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:04.471693 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:04.471696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:04.471697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:04.471903 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:04.472002 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:04.472478 (+ 476us) server_negotiation.cc:299] Negotiation successful
0806 16:17:04.472687 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":64}
I20260806 16:17:04.472862 15748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.462769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:36821 (local address 127.0.0.1:33014)
0806 16:17:04.463148 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:04.463163 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:04.463295 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:04.463699 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:04.463702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:04.464382 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:04.464531 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.464539 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.465519 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.465522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.466121 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.466130 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.466646 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.466650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.466722 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.467457 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:04.467470 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:04.469105 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:04.470947 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:04.470956 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:04.470963 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:04.471222 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:04.471508 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:04.471511 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:04.471512 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:04.471598 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:04.472056 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:04.472067 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:04.472252 (+ 185us) client_negotiation.cc:769] Sending connection context
0806 16:17:04.472419 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 16:17:04.472605 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":154,"threads_started":1}
W20260806 16:17:04.475958 15272 tablet.cc:2369] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:17:04.478916 14958 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:33014:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 16:17:04.479909 14958 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:17:04.486821 15181 tablet_service.cc:1467] Processing CreateTablet for tablet b9082a1d95ac46e5963ed5f190570ae8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4649d49cfe4641cab8b05488c39b05ff]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:17:04.486953 15372 tablet_service.cc:1467] Processing CreateTablet for tablet b9082a1d95ac46e5963ed5f190570ae8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4649d49cfe4641cab8b05488c39b05ff]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:17:04.487800 15181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9082a1d95ac46e5963ed5f190570ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:04.487807 15372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9082a1d95ac46e5963ed5f190570ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:04.491508 15725 tablet_bootstrap.cc:492] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28: Bootstrap starting.
I20260806 16:17:04.491667 15727 tablet_bootstrap.cc:492] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08: Bootstrap starting.
I20260806 16:17:04.492172 15559 tablet_service.cc:1467] Processing CreateTablet for tablet b9082a1d95ac46e5963ed5f190570ae8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4649d49cfe4641cab8b05488c39b05ff]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:17:04.492905 15727 tablet_bootstrap.cc:654] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:04.493222 15559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9082a1d95ac46e5963ed5f190570ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:04.493221 15725 tablet_bootstrap.cc:654] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:04.494915 15727 tablet_bootstrap.cc:492] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08: No bootstrap required, opened a new log
I20260806 16:17:04.495044 15727 ts_tablet_manager.cc:1397] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:04.495324 15725 tablet_bootstrap.cc:492] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28: No bootstrap required, opened a new log
I20260806 16:17:04.495476 15725 ts_tablet_manager.cc:1397] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260806 16:17:04.496096 15727 raft_consensus.cc:348] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:04.496048 15725 raft_consensus.cc:348] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:04.496268 15727 raft_consensus.cc:374] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:04.496281 15725 raft_consensus.cc:374] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:04.496317 15727 raft_consensus.cc:729] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e54dd8327a4d48a27207bc73dbfd08, State: Initialized, Role: FOLLOWER
I20260806 16:17:04.496325 15725 raft_consensus.cc:729] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945ddb389c3f48b6ab135573c825de28, State: Initialized, Role: FOLLOWER
I20260806 16:17:04.496460 15727 consensus_queue.cc:260] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:04.496523 15725 consensus_queue.cc:260] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:04.496758 15727 ts_tablet_manager.cc:1428] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 16:17:04.496876 15734 tablet_replica.cc:439] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } }
I20260806 16:17:04.497036 15734 tablet_replica.cc:442] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:04.497068 15732 tablet_replica.cc:439] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } }
I20260806 16:17:04.497218 15732 tablet_replica.cc:442] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:04.497280 15722 tablet_bootstrap.cc:492] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236: Bootstrap starting.
I20260806 16:17:04.497939 15725 ts_tablet_manager.cc:1428] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 16:17:04.498489 15722 tablet_bootstrap.cc:654] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:04.500382 15722 tablet_bootstrap.cc:492] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236: No bootstrap required, opened a new log
I20260806 16:17:04.500499 15722 ts_tablet_manager.cc:1397] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:04.500982 15722 raft_consensus.cc:348] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:04.501088 15722 raft_consensus.cc:374] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:04.501111 15722 raft_consensus.cc:729] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349bce4fe2e7407bb3ad0407b5a8e236, State: Initialized, Role: FOLLOWER
I20260806 16:17:04.501281 15722 consensus_queue.cc:260] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:04.501595 15722 ts_tablet_manager.cc:1428] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 16:17:04.501574 15733 tablet_replica.cc:439] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } }
I20260806 16:17:04.501827 15733 tablet_replica.cc:442] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:04.734203 15733 raft_consensus.cc:482] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:04.734444 15733 raft_consensus.cc:504] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:04.736529 15733 leader_election.cc:288] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 945ddb389c3f48b6ab135573c825de28 (127.11.245.129:39757), 12e54dd8327a4d48a27207bc73dbfd08 (127.11.245.130:44595)
I20260806 16:17:04.741976 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.738005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:51627 (local address 127.11.245.130:44595)
0806 16:17:04.738188 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:04.738193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:04.738211 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:17:04.738283 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:04.738285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:04.738340 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:04.738458 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:04.738856 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.739557 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.740883 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.741328 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.741435 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.741527 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:04.741718 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 16:17:04.741816 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 16:17:04.741991 15752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.736928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:39757 (local address 127.11.245.131:44627)
0806 16:17:04.737287 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:04.737306 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:04.737466 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:04.738857 (+ 1391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:04.738860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:04.738882 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:04.739010 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.739018 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.739877 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.739879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.740762 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.740772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.741531 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.741536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.741633 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.741671 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 16:17:04.741754 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 16:17:04.741834 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":22,"thread_start_us":170,"threads_started":1}
I20260806 16:17:04.742007 15753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.737324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:44595 (local address 127.11.245.131:51627)
0806 16:17:04.737868 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:04.737889 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:04.738023 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:04.738504 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:04.738508 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:04.738534 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:04.738699 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.738711 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.739676 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.739680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.740747 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.740757 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.741460 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.741467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.741571 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.741597 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 16:17:04.741678 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 16:17:04.741784 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":1,"thread_start_us":132,"threads_started":1}
I20260806 16:17:04.742035 15754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.738205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:44627 (local address 127.11.245.129:39757)
0806 16:17:04.738504 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:04.738510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:04.738524 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:17:04.738596 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:04.738599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:04.738660 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:04.738772 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:04.739137 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.739748 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.740931 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.741370 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.741532 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.741596 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:04.741791 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 16:17:04.741887 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":81,"threads_started":1}
I20260806 16:17:04.742612 15394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9082a1d95ac46e5963ed5f190570ae8" candidate_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12e54dd8327a4d48a27207bc73dbfd08" is_pre_election: true
I20260806 16:17:04.742864 15394 raft_consensus.cc:2454] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349bce4fe2e7407bb3ad0407b5a8e236 in term 0.
I20260806 16:17:04.743021 15210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9082a1d95ac46e5963ed5f190570ae8" candidate_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945ddb389c3f48b6ab135573c825de28" is_pre_election: true
I20260806 16:17:04.743363 15210 raft_consensus.cc:2454] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349bce4fe2e7407bb3ad0407b5a8e236 in term 0.
I20260806 16:17:04.743331 15514 leader_election.cc:302] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12e54dd8327a4d48a27207bc73dbfd08, 349bce4fe2e7407bb3ad0407b5a8e236; no voters:
I20260806 16:17:04.743754 15733 raft_consensus.cc:2781] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:04.743853 15733 raft_consensus.cc:482] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:04.743902 15733 raft_consensus.cc:3037] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:04.745479 15733 raft_consensus.cc:504] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:04.745911 15733 leader_election.cc:288] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [CANDIDATE]: Term 1 election: Requested vote from peers 945ddb389c3f48b6ab135573c825de28 (127.11.245.129:39757), 12e54dd8327a4d48a27207bc73dbfd08 (127.11.245.130:44595)
I20260806 16:17:04.746352 15210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9082a1d95ac46e5963ed5f190570ae8" candidate_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945ddb389c3f48b6ab135573c825de28"
I20260806 16:17:04.746371 15394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9082a1d95ac46e5963ed5f190570ae8" candidate_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12e54dd8327a4d48a27207bc73dbfd08"
I20260806 16:17:04.746515 15210 raft_consensus.cc:3037] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:04.746526 15394 raft_consensus.cc:3037] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:04.747751 15210 raft_consensus.cc:2454] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349bce4fe2e7407bb3ad0407b5a8e236 in term 1.
I20260806 16:17:04.747810 15394 raft_consensus.cc:2454] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349bce4fe2e7407bb3ad0407b5a8e236 in term 1.
I20260806 16:17:04.748195 15514 leader_election.cc:302] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12e54dd8327a4d48a27207bc73dbfd08, 349bce4fe2e7407bb3ad0407b5a8e236; no voters:
I20260806 16:17:04.748559 15733 raft_consensus.cc:2781] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:04.748797 15733 raft_consensus.cc:686] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 LEADER]: Becoming Leader. State: Replica: 349bce4fe2e7407bb3ad0407b5a8e236, State: Running, Role: LEADER
I20260806 16:17:04.749354 15733 consensus_queue.cc:237] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:04.752507 15755 tablet_replica.cc:439] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236: TxnStatusTablet state changed. Reason: New leader 349bce4fe2e7407bb3ad0407b5a8e236. Latest consensus state: current_term: 1 leader_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } }
I20260806 16:17:04.752645 15755 tablet_replica.cc:442] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:17:04.753013 15758 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 16:17:04.753341 15758 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 16:17:04.754307 14958 catalog_manager.cc:5495] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 reported cstate change: term changed from 0 to 1, leader changed from <none> to 349bce4fe2e7407bb3ad0407b5a8e236 (127.11.245.131). New cstate: current_term: 1 leader_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } health_report { overall_health: HEALTHY } } }
I20260806 16:17:04.798779 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.793944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54246 (local address 127.11.245.131:35401)
0806 16:17:04.794086 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:04.794091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:04.794107 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:17:04.794348 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:04.794352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:04.794433 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:04.794532 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:04.794873 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.795706 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.797028 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.797373 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.797476 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.797852 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:04.798365 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:04.798460 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:04.798583 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 16:17:04.798639 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 16:17:04.798815 15748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.793797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:35401 (local address 127.0.0.1:54246)
0806 16:17:04.793985 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:04.794005 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:04.794141 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:04.794586 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:04.794590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:04.794600 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:17:04.794747 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.794754 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.795834 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.795838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.796885 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.796896 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.797497 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.797501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.797599 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.797668 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:17:04.798461 (+ 793us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:17:04.798478 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 16:17:04.798556 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 16:17:04.798656 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 16:17:04.805583 15394 raft_consensus.cc:1261] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 FOLLOWER]: Refusing update from remote peer 349bce4fe2e7407bb3ad0407b5a8e236: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:04.805583 15210 raft_consensus.cc:1261] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28 [term 1 FOLLOWER]: Refusing update from remote peer 349bce4fe2e7407bb3ad0407b5a8e236: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:04.806380 15755 consensus_queue.cc:1035] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [LEADER]: Connected to new peer: Peer: permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:04.806619 15733 consensus_queue.cc:1035] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:04.810460 15734 tablet_replica.cc:439] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28: TxnStatusTablet state changed. Reason: New leader 349bce4fe2e7407bb3ad0407b5a8e236. Latest consensus state: current_term: 1 leader_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } }
I20260806 16:17:04.810622 15734 tablet_replica.cc:442] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:04.814937 15755 tablet_replica.cc:439] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } }
I20260806 16:17:04.815099 15755 tablet_replica.cc:442] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:17:04.815426 15734 consensus_queue.cc:1035] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:04.816886 15732 tablet_replica.cc:439] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08: TxnStatusTablet state changed. Reason: New leader 349bce4fe2e7407bb3ad0407b5a8e236. Latest consensus state: current_term: 1 leader_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } }
I20260806 16:17:04.817066 15732 tablet_replica.cc:442] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:04.820971 15755 tablet_replica.cc:439] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } }
I20260806 16:17:04.821133 15755 tablet_replica.cc:442] T b9082a1d95ac46e5963ed5f190570ae8 P 349bce4fe2e7407bb3ad0407b5a8e236: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:17:04.822788 15737 tablet_replica.cc:439] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } }
I20260806 16:17:04.822979 15737 tablet_replica.cc:442] T b9082a1d95ac46e5963ed5f190570ae8 P 945ddb389c3f48b6ab135573c825de28: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:04.823187 15765 mvcc.cc:205] Tried to move back new op lower bound from 7315591269594640384 to 7315591269377183744. Current Snapshot: MvccSnapshot[applied={T|T < 7315591269594640384}]
I20260806 16:17:04.823347 15732 tablet_replica.cc:439] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } } }
I20260806 16:17:04.823487 15732 tablet_replica.cc:442] T b9082a1d95ac46e5963ed5f190570ae8 P 12e54dd8327a4d48a27207bc73dbfd08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:04.823459 15767 mvcc.cc:205] Tried to move back new op lower bound from 7315591269594640384 to 7315591269377183744. Current Snapshot: MvccSnapshot[applied={T|T < 7315591269594640384}]
I20260806 16:17:04.825229 15734 consensus_queue.cc:1035] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:04.833503 15754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.827785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59142 (local address 127.11.245.129:39757)
0806 16:17:04.827937 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:04.827942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:04.828469 (+ 527us) server_negotiation.cc:407] Connection header received
0806 16:17:04.828724 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:04.828727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:04.828803 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:04.828895 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:04.829366 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.830127 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.831418 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.831738 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.831818 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.832302 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:04.832970 (+ 668us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:04.833115 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:04.833255 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 16:17:04.833337 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 16:17:04.848214 15787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.843286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:35401 (local address 127.11.245.129:48705)
0806 16:17:04.843796 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:04.843812 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:04.843977 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:04.844335 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:04.844338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:04.844345 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:17:04.844504 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.844510 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.845600 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.845603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.846411 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.846422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.847197 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.847203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.847298 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.847358 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:17:04.847776 (+ 418us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:17:04.847784 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 16:17:04.847869 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 16:17:04.847973 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":183,"threads_started":1}
I20260806 16:17:04.848197 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.843441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:48705 (local address 127.11.245.131:35401)
0806 16:17:04.843629 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:04.843635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:04.843914 (+ 279us) server_negotiation.cc:407] Connection header received
0806 16:17:04.844125 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:04.844128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:04.844194 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:04.844306 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:04.844623 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.845494 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.846633 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.847013 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.847120 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.847558 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:04.847681 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:04.847756 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:04.847936 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 16:17:04.848003 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 16:17:04.888688 15754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.883667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:52829 (local address 127.11.245.129:39757)
0806 16:17:04.883803 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:04.883809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:04.884498 (+ 689us) server_negotiation.cc:407] Connection header received
0806 16:17:04.884734 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:04.884737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:04.884809 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:04.884904 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:04.885260 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.885864 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.887212 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.887522 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.887648 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.888009 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:04.888170 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:04.888235 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:04.888391 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 16:17:04.888499 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 16:17:04.889619 15794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:04.883596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:39757 (local address 127.11.245.131:52829)
0806 16:17:04.884373 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:04.884395 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:04.884562 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:04.884923 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:04.884927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:04.884938 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:17:04.885108 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.885116 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.886011 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.886015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.887075 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:04.887088 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:04.887671 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:04.887675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:04.887778 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:04.887843 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:17:04.888259 (+ 416us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:17:04.888266 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 16:17:04.888986 (+ 720us) client_negotiation.cc:240] Negotiation successful
0806 16:17:04.889098 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":135,"threads_started":1}
I20260806 16:17:04.930683 14958 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:04.931007 14958 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:04.935232 14958 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d: Sending DeleteTablet for 3 replicas of tablet 754da61692b34f00951dc49998a1141e
I20260806 16:17:04.936602 15181 tablet_service.cc:1514] Processing DeleteTablet for tablet 754da61692b34f00951dc49998a1141e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:04 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:59132
I20260806 16:17:04.937311 15559 tablet_service.cc:1514] Processing DeleteTablet for tablet 754da61692b34f00951dc49998a1141e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:04 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:54244
I20260806 16:17:04.937405 15798 tablet_replica.cc:330] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28: stopping tablet replica
I20260806 16:17:04.938093 15798 raft_consensus.cc:2229] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:17:04.938431 15372 tablet_service.cc:1514] Processing DeleteTablet for tablet 754da61692b34f00951dc49998a1141e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:04 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:41322
I20260806 16:17:04.938941 15798 raft_consensus.cc:2258] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:04.940613 14958 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:17:04.941254 14958 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:17:04.941272 15798 ts_tablet_manager.cc:1905] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:04.945858 15798 ts_tablet_manager.cc:1918] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:17:04.946175 15798 log.cc:1196] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/wal/wals/754da61692b34f00951dc49998a1141e
I20260806 16:17:04.946705 15798 ts_tablet_manager.cc:1939] T 754da61692b34f00951dc49998a1141e P 945ddb389c3f48b6ab135573c825de28: Deleting consensus metadata
I20260806 16:17:04.948173 15181 tablet_service.cc:1467] Processing CreateTablet for tablet 84c159cc509d48749b05bb64430ce230 (DEFAULT_TABLE table=test-table [id=d1a6cc5381d84bd28fe800f2af50a8cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:04.948508 15181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84c159cc509d48749b05bb64430ce230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:04.948561 14934 catalog_manager.cc:4841] TS 945ddb389c3f48b6ab135573c825de28 (127.11.245.129:39757): tablet 754da61692b34f00951dc49998a1141e (table test-table [id=bd74152e80054e50ba4be5985571f8fb]) successfully deleted
I20260806 16:17:04.951979 15725 tablet_bootstrap.cc:492] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28: Bootstrap starting.
I20260806 16:17:04.952513 15559 tablet_service.cc:1467] Processing CreateTablet for tablet 84c159cc509d48749b05bb64430ce230 (DEFAULT_TABLE table=test-table [id=d1a6cc5381d84bd28fe800f2af50a8cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:04.952922 15559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84c159cc509d48749b05bb64430ce230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:04.953312 15725 tablet_bootstrap.cc:654] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:04.954892 15725 tablet_bootstrap.cc:492] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28: No bootstrap required, opened a new log
I20260806 16:17:04.955005 15725 ts_tablet_manager.cc:1397] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 16:17:04.955345 15800 tablet_replica.cc:330] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08: stopping tablet replica
I20260806 16:17:04.955394 15372 tablet_service.cc:1467] Processing CreateTablet for tablet 84c159cc509d48749b05bb64430ce230 (DEFAULT_TABLE table=test-table [id=d1a6cc5381d84bd28fe800f2af50a8cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:04.955586 15725 raft_consensus.cc:348] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:04.955749 15372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84c159cc509d48749b05bb64430ce230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:04.955751 15725 raft_consensus.cc:374] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:04.955793 15725 raft_consensus.cc:729] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945ddb389c3f48b6ab135573c825de28, State: Initialized, Role: FOLLOWER
I20260806 16:17:04.955904 15800 raft_consensus.cc:2229] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:04.955968 15725 consensus_queue.cc:260] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:04.956288 15800 raft_consensus.cc:2258] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:04.956434 15725 ts_tablet_manager.cc:1428] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260806 16:17:04.956815 15799 tablet_replica.cc:330] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236: stopping tablet replica
I20260806 16:17:04.957181 15799 raft_consensus.cc:2229] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:04.957384 15799 raft_consensus.cc:2258] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:04.959085 15799 ts_tablet_manager.cc:1905] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:04.959085 15800 ts_tablet_manager.cc:1905] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:04.959837 15727 tablet_bootstrap.cc:492] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08: Bootstrap starting.
I20260806 16:17:04.961030 15727 tablet_bootstrap.cc:654] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:04.962256 15800 ts_tablet_manager.cc:1918] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:17:04.962358 15800 log.cc:1196] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/754da61692b34f00951dc49998a1141e
I20260806 16:17:04.962558 15727 tablet_bootstrap.cc:492] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08: No bootstrap required, opened a new log
I20260806 16:17:04.962644 15727 ts_tablet_manager.cc:1397] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:04.962725 15800 ts_tablet_manager.cc:1939] T 754da61692b34f00951dc49998a1141e P 12e54dd8327a4d48a27207bc73dbfd08: Deleting consensus metadata
I20260806 16:17:04.962965 15799 ts_tablet_manager.cc:1918] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:17:04.963074 15799 log.cc:1196] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/754da61692b34f00951dc49998a1141e
I20260806 16:17:04.963222 15727 raft_consensus.cc:348] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:04.963343 15727 raft_consensus.cc:374] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:04.963373 15727 raft_consensus.cc:729] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e54dd8327a4d48a27207bc73dbfd08, State: Initialized, Role: FOLLOWER
I20260806 16:17:04.963392 15799 ts_tablet_manager.cc:1939] T 754da61692b34f00951dc49998a1141e P 349bce4fe2e7407bb3ad0407b5a8e236: Deleting consensus metadata
I20260806 16:17:04.963513 15727 consensus_queue.cc:260] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:04.964313 14936 catalog_manager.cc:4841] TS 12e54dd8327a4d48a27207bc73dbfd08 (127.11.245.130:44595): tablet 754da61692b34f00951dc49998a1141e (table test-table [id=bd74152e80054e50ba4be5985571f8fb]) successfully deleted
I20260806 16:17:04.964670 15727 ts_tablet_manager.cc:1428] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 16:17:04.965492 14934 catalog_manager.cc:4841] TS 349bce4fe2e7407bb3ad0407b5a8e236 (127.11.245.131:35401): tablet 754da61692b34f00951dc49998a1141e (table test-table [id=bd74152e80054e50ba4be5985571f8fb]) successfully deleted
I20260806 16:17:04.966048 15722 tablet_bootstrap.cc:492] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236: Bootstrap starting.
I20260806 16:17:04.967103 15722 tablet_bootstrap.cc:654] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:04.968436 15722 tablet_bootstrap.cc:492] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236: No bootstrap required, opened a new log
I20260806 16:17:04.968518 15722 ts_tablet_manager.cc:1397] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:04.969058 15722 raft_consensus.cc:348] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:04.969202 15722 raft_consensus.cc:374] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:04.969249 15722 raft_consensus.cc:729] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349bce4fe2e7407bb3ad0407b5a8e236, State: Initialized, Role: FOLLOWER
I20260806 16:17:04.969408 15722 consensus_queue.cc:260] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:04.969851 15722 ts_tablet_manager.cc:1428] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 16:17:05.045024 15733 raft_consensus.cc:482] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:05.045231 15733 raft_consensus.cc:504] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:05.045933 15733 leader_election.cc:288] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 945ddb389c3f48b6ab135573c825de28 (127.11.245.129:39757), 12e54dd8327a4d48a27207bc73dbfd08 (127.11.245.130:44595)
I20260806 16:17:05.046553 15210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c159cc509d48749b05bb64430ce230" candidate_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945ddb389c3f48b6ab135573c825de28" is_pre_election: true
I20260806 16:17:05.046556 15394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c159cc509d48749b05bb64430ce230" candidate_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12e54dd8327a4d48a27207bc73dbfd08" is_pre_election: true
I20260806 16:17:05.046767 15210 raft_consensus.cc:2454] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349bce4fe2e7407bb3ad0407b5a8e236 in term 0.
I20260806 16:17:05.046790 15394 raft_consensus.cc:2454] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349bce4fe2e7407bb3ad0407b5a8e236 in term 0.
I20260806 16:17:05.047192 15511 leader_election.cc:302] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349bce4fe2e7407bb3ad0407b5a8e236, 945ddb389c3f48b6ab135573c825de28; no voters:
I20260806 16:17:05.047526 15733 raft_consensus.cc:2781] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:05.047611 15733 raft_consensus.cc:482] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:05.047669 15733 raft_consensus.cc:3037] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:05.049147 15733 raft_consensus.cc:504] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:05.049608 15733 leader_election.cc:288] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [CANDIDATE]: Term 1 election: Requested vote from peers 945ddb389c3f48b6ab135573c825de28 (127.11.245.129:39757), 12e54dd8327a4d48a27207bc73dbfd08 (127.11.245.130:44595)
I20260806 16:17:05.050067 15210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c159cc509d48749b05bb64430ce230" candidate_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945ddb389c3f48b6ab135573c825de28"
I20260806 16:17:05.050138 15394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84c159cc509d48749b05bb64430ce230" candidate_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12e54dd8327a4d48a27207bc73dbfd08"
I20260806 16:17:05.050233 15210 raft_consensus.cc:3037] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:05.050304 15394 raft_consensus.cc:3037] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:05.051623 15210 raft_consensus.cc:2454] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349bce4fe2e7407bb3ad0407b5a8e236 in term 1.
I20260806 16:17:05.051715 15394 raft_consensus.cc:2454] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349bce4fe2e7407bb3ad0407b5a8e236 in term 1.
I20260806 16:17:05.052026 15511 leader_election.cc:302] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349bce4fe2e7407bb3ad0407b5a8e236, 945ddb389c3f48b6ab135573c825de28; no voters:
I20260806 16:17:05.052251 15733 raft_consensus.cc:2781] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:05.052358 15733 raft_consensus.cc:686] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 LEADER]: Becoming Leader. State: Replica: 349bce4fe2e7407bb3ad0407b5a8e236, State: Running, Role: LEADER
I20260806 16:17:05.052544 15733 consensus_queue.cc:237] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:05.055109 14958 catalog_manager.cc:5495] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 reported cstate change: term changed from 0 to 1, leader changed from <none> to 349bce4fe2e7407bb3ad0407b5a8e236 (127.11.245.131). New cstate: current_term: 1 leader_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } health_report { overall_health: HEALTHY } } }
I20260806 16:17:05.091759 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:05.088152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54262 (local address 127.11.245.131:35401)
0806 16:17:05.088520 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:05.088524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:05.088535 (+ 11us) server_negotiation.cc:407] Connection header received
0806 16:17:05.088581 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:05.088583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:05.088632 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:05.088724 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:05.089030 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:05.089597 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:05.090520 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:05.090742 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:05.090861 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:05.091176 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:05.091309 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:05.091384 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:05.091490 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 16:17:05.091565 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271}
I20260806 16:17:05.104741 15794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:05.100938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:35401 (local address 127.11.245.131:41471)
0806 16:17:05.101141 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:05.101160 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:05.101281 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:05.101572 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:05.101575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:05.101581 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:17:05.101888 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:05.101902 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:05.102790 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:05.102796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:05.103415 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:05.103423 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:05.103878 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:05.103882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:05.103965 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:05.103991 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:17:05.104346 (+ 355us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:17:05.104354 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 16:17:05.104451 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 16:17:05.104565 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":3}
I20260806 16:17:05.104756 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:05.101065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:41471 (local address 127.11.245.131:35401)
0806 16:17:05.101186 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:05.101193 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:05.101226 (+ 33us) server_negotiation.cc:407] Connection header received
0806 16:17:05.101428 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:05.101430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:05.101475 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:05.101553 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:05.102098 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:05.102676 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:05.103541 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:05.103756 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:05.103876 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:05.104140 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:05.104254 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:05.104329 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:05.104503 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 16:17:05.104574 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":60}
I20260806 16:17:05.112974 15394 raft_consensus.cc:1261] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 FOLLOWER]: Refusing update from remote peer 349bce4fe2e7407bb3ad0407b5a8e236: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:05.113246 15210 raft_consensus.cc:1261] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28 [term 1 FOLLOWER]: Refusing update from remote peer 349bce4fe2e7407bb3ad0407b5a8e236: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:05.113660 15733 consensus_queue.cc:1035] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:05.113893 15755 consensus_queue.cc:1035] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [LEADER]: Connected to new peer: Peer: permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:05.123340 15764 mvcc.cc:205] Tried to move back new op lower bound from 7315591270857883648 to 7315591270618292224. Current Snapshot: MvccSnapshot[applied={T|T < 7315591270857883648}]
W20260806 16:17:05.149142 15764 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 16:17:06.731360 14959 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:06.731632 14959 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:06.735778 14959 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d: Sending DeleteTablet for 3 replicas of tablet 84c159cc509d48749b05bb64430ce230
I20260806 16:17:06.736937 15181 tablet_service.cc:1514] Processing DeleteTablet for tablet 84c159cc509d48749b05bb64430ce230 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:06 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:59132
I20260806 16:17:06.737178 15372 tablet_service.cc:1514] Processing DeleteTablet for tablet 84c159cc509d48749b05bb64430ce230 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:06 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:41322
I20260806 16:17:06.737390 15842 tablet_replica.cc:330] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28: stopping tablet replica
I20260806 16:17:06.737581 15842 raft_consensus.cc:2229] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:06.737797 15842 raft_consensus.cc:2258] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:06.738271 15844 tablet_replica.cc:330] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08: stopping tablet replica
I20260806 16:17:06.738526 15844 raft_consensus.cc:2229] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:06.738708 15844 raft_consensus.cc:2258] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:06.738785 15842 ts_tablet_manager.cc:1905] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:06.740039 15844 ts_tablet_manager.cc:1905] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:06.742815 15842 ts_tablet_manager.cc:1918] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 16:17:06.742848 15562 tablet_service.cc:1514] Processing DeleteTablet for tablet 84c159cc509d48749b05bb64430ce230 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:06 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:54244
I20260806 16:17:06.742959 15842 log.cc:1196] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/wal/wals/84c159cc509d48749b05bb64430ce230
I20260806 16:17:06.743292 15842 ts_tablet_manager.cc:1939] T 84c159cc509d48749b05bb64430ce230 P 945ddb389c3f48b6ab135573c825de28: Deleting consensus metadata
I20260806 16:17:06.743327 15847 tablet_replica.cc:330] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236: stopping tablet replica
I20260806 16:17:06.743620 15847 raft_consensus.cc:2229] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:17:06.743743 15844 ts_tablet_manager.cc:1918] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 16:17:06.743906 15844 log.cc:1196] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/84c159cc509d48749b05bb64430ce230
I20260806 16:17:06.744248 15847 raft_consensus.cc:2258] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:06.744315 15844 ts_tablet_manager.cc:1939] T 84c159cc509d48749b05bb64430ce230 P 12e54dd8327a4d48a27207bc73dbfd08: Deleting consensus metadata
I20260806 16:17:06.744462 14934 catalog_manager.cc:4841] TS 945ddb389c3f48b6ab135573c825de28 (127.11.245.129:39757): tablet 84c159cc509d48749b05bb64430ce230 (table test-table [id=d1a6cc5381d84bd28fe800f2af50a8cc]) successfully deleted
I20260806 16:17:06.745229 15847 ts_tablet_manager.cc:1905] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:06.745307 14936 catalog_manager.cc:4841] TS 12e54dd8327a4d48a27207bc73dbfd08 (127.11.245.130:44595): tablet 84c159cc509d48749b05bb64430ce230 (table test-table [id=d1a6cc5381d84bd28fe800f2af50a8cc]) successfully deleted
I20260806 16:17:06.748687 15847 ts_tablet_manager.cc:1918] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 16:17:06.748798 15847 log.cc:1196] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/84c159cc509d48749b05bb64430ce230
I20260806 16:17:06.749125 15847 ts_tablet_manager.cc:1939] T 84c159cc509d48749b05bb64430ce230 P 349bce4fe2e7407bb3ad0407b5a8e236: Deleting consensus metadata
I20260806 16:17:06.750587 14934 catalog_manager.cc:4841] TS 349bce4fe2e7407bb3ad0407b5a8e236 (127.11.245.131:35401): tablet 84c159cc509d48749b05bb64430ce230 (table test-table [id=d1a6cc5381d84bd28fe800f2af50a8cc]) successfully deleted
Aug 06 16:17:06 dist-test-slave-t3q3 krb5kdc[14887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033026, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 16:17:06 dist-test-slave-t3q3 krb5kdc[14887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033026, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:06.779237 15859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:06.763791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33016 (local address 127.11.245.190:36821)
0806 16:17:06.764281 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:06.764291 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:06.764309 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:17:06.764375 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:06.764378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:06.764434 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:06.764536 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:06.765371 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:06.766077 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:06.767044 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:06.767291 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:06.767383 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:06.775182 (+ 7799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:06.775187 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:06.775189 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:06.775234 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:06.777143 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:06.777697 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:06.777701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:06.777703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:06.777818 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:06.778152 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:06.778156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:06.778158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:06.778413 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:06.778571 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:06.778847 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 16:17:06.779003 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":165,"threads_started":1}
I20260806 16:17:06.782925 14959 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33016:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:17:06.783586 14959 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:17:06.789640 15181 tablet_service.cc:1467] Processing CreateTablet for tablet d5e5ef074b934a379492ee806a7fb5e3 (DEFAULT_TABLE table=test-table [id=91819665bde74bb48a6cb1d96d583682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:06.789839 15562 tablet_service.cc:1467] Processing CreateTablet for tablet d5e5ef074b934a379492ee806a7fb5e3 (DEFAULT_TABLE table=test-table [id=91819665bde74bb48a6cb1d96d583682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:06.789849 15372 tablet_service.cc:1467] Processing CreateTablet for tablet d5e5ef074b934a379492ee806a7fb5e3 (DEFAULT_TABLE table=test-table [id=91819665bde74bb48a6cb1d96d583682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:06.790104 15181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e5ef074b934a379492ee806a7fb5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:06.790206 15562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e5ef074b934a379492ee806a7fb5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:06.790215 15372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e5ef074b934a379492ee806a7fb5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:06.793771 15862 tablet_bootstrap.cc:492] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28: Bootstrap starting.
I20260806 16:17:06.793797 15861 tablet_bootstrap.cc:492] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236: Bootstrap starting.
I20260806 16:17:06.794910 15862 tablet_bootstrap.cc:654] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:06.794915 15861 tablet_bootstrap.cc:654] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:06.796334 15860 tablet_bootstrap.cc:492] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08: Bootstrap starting.
I20260806 16:17:06.796535 15862 tablet_bootstrap.cc:492] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28: No bootstrap required, opened a new log
I20260806 16:17:06.796640 15862 ts_tablet_manager.cc:1397] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:06.797246 15861 tablet_bootstrap.cc:492] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236: No bootstrap required, opened a new log
I20260806 16:17:06.797416 15861 ts_tablet_manager.cc:1397] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:17:06.797384 15862 raft_consensus.cc:348] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:06.797627 15862 raft_consensus.cc:374] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:06.797709 15862 raft_consensus.cc:729] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945ddb389c3f48b6ab135573c825de28, State: Initialized, Role: FOLLOWER
I20260806 16:17:06.797919 15862 consensus_queue.cc:260] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:06.798010 15861 raft_consensus.cc:348] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:06.798134 15861 raft_consensus.cc:374] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:06.798189 15861 raft_consensus.cc:729] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349bce4fe2e7407bb3ad0407b5a8e236, State: Initialized, Role: FOLLOWER
I20260806 16:17:06.798548 15862 ts_tablet_manager.cc:1428] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 16:17:06.798481 15861 consensus_queue.cc:260] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:06.798871 15861 ts_tablet_manager.cc:1428] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:17:06.800387 15860 tablet_bootstrap.cc:654] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:06.802095 15860 tablet_bootstrap.cc:492] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08: No bootstrap required, opened a new log
I20260806 16:17:06.802464 15860 ts_tablet_manager.cc:1397] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260806 16:17:06.803138 15860 raft_consensus.cc:348] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:06.803298 15860 raft_consensus.cc:374] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:06.803344 15860 raft_consensus.cc:729] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e54dd8327a4d48a27207bc73dbfd08, State: Initialized, Role: FOLLOWER
I20260806 16:17:06.803510 15860 consensus_queue.cc:260] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:06.804126 15860 ts_tablet_manager.cc:1428] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 16:17:06.916698 15872 raft_consensus.cc:482] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:06.917042 15872 raft_consensus.cc:504] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:06.919288 15872 leader_election.cc:288] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 945ddb389c3f48b6ab135573c825de28 (127.11.245.129:39757), 349bce4fe2e7407bb3ad0407b5a8e236 (127.11.245.131:35401)
I20260806 16:17:06.926306 15881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:06.920309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:34931 (local address 127.11.245.131:35401)
0806 16:17:06.920691 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:06.920706 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:06.920725 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:17:06.920799 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:06.920803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:06.920864 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:06.920981 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:06.921686 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:06.922654 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:06.924768 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:06.925464 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:06.925627 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:06.925721 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:06.925998 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 16:17:06.926091 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":170,"threads_started":1}
I20260806 16:17:06.926311 15878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:06.920126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:35401 (local address 127.11.245.130:34931)
0806 16:17:06.920397 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:06.920414 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:06.920535 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:06.921001 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:06.921004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:06.921025 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:06.921199 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:06.921209 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:06.922761 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:06.922764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:06.924600 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:06.924615 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:06.925634 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:06.925640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:06.925825 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:06.925866 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 16:17:06.925969 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 16:17:06.926094 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":198,"thread_start_us":107,"threads_started":1}
I20260806 16:17:06.927058 15874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:06.919589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:39757 (local address 127.11.245.130:42325)
0806 16:17:06.920134 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:06.920156 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:06.920330 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:06.921677 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:06.921680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:06.921699 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:06.921848 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:06.921854 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:06.922704 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:06.922708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:06.924603 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:06.924615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:06.925630 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:06.925635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:06.926601 (+ 966us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:06.926640 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 16:17:06.926743 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 16:17:06.926864 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":1001,"thread_start_us":167,"threads_started":1}
I20260806 16:17:06.927076 15882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:06.920592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:42325 (local address 127.11.245.129:39757)
0806 16:17:06.920931 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:06.921310 (+ 379us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:06.921333 (+ 23us) server_negotiation.cc:407] Connection header received
0806 16:17:06.921409 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:06.921412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:06.921505 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:06.921651 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:06.921968 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:06.922560 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:06.924769 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:06.925475 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:06.925815 (+ 340us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:06.925866 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:06.926773 (+ 907us) server_negotiation.cc:299] Negotiation successful
0806 16:17:06.926905 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":158,"threads_started":1}
I20260806 16:17:06.927245 15587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e5ef074b934a379492ee806a7fb5e3" candidate_uuid: "12e54dd8327a4d48a27207bc73dbfd08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" is_pre_election: true
I20260806 16:17:06.927476 15587 raft_consensus.cc:2454] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12e54dd8327a4d48a27207bc73dbfd08 in term 0.
I20260806 16:17:06.927829 15210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e5ef074b934a379492ee806a7fb5e3" candidate_uuid: "12e54dd8327a4d48a27207bc73dbfd08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945ddb389c3f48b6ab135573c825de28" is_pre_election: true
I20260806 16:17:06.927996 15210 raft_consensus.cc:2454] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12e54dd8327a4d48a27207bc73dbfd08 in term 0.
I20260806 16:17:06.928565 15303 leader_election.cc:302] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12e54dd8327a4d48a27207bc73dbfd08, 945ddb389c3f48b6ab135573c825de28; no voters:
I20260806 16:17:06.928957 15872 raft_consensus.cc:2781] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:06.929039 15872 raft_consensus.cc:482] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:06.929080 15872 raft_consensus.cc:3037] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:06.930790 15872 raft_consensus.cc:504] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:06.931355 15872 leader_election.cc:288] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [CANDIDATE]: Term 1 election: Requested vote from peers 945ddb389c3f48b6ab135573c825de28 (127.11.245.129:39757), 349bce4fe2e7407bb3ad0407b5a8e236 (127.11.245.131:35401)
I20260806 16:17:06.931959 15587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e5ef074b934a379492ee806a7fb5e3" candidate_uuid: "12e54dd8327a4d48a27207bc73dbfd08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349bce4fe2e7407bb3ad0407b5a8e236"
I20260806 16:17:06.931988 15210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e5ef074b934a379492ee806a7fb5e3" candidate_uuid: "12e54dd8327a4d48a27207bc73dbfd08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945ddb389c3f48b6ab135573c825de28"
I20260806 16:17:06.932166 15210 raft_consensus.cc:3037] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:06.932166 15587 raft_consensus.cc:3037] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:06.933719 15210 raft_consensus.cc:2454] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12e54dd8327a4d48a27207bc73dbfd08 in term 1.
I20260806 16:17:06.933729 15587 raft_consensus.cc:2454] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12e54dd8327a4d48a27207bc73dbfd08 in term 1.
I20260806 16:17:06.934142 15303 leader_election.cc:302] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12e54dd8327a4d48a27207bc73dbfd08, 349bce4fe2e7407bb3ad0407b5a8e236; no voters:
I20260806 16:17:06.934404 15872 raft_consensus.cc:2781] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:06.934681 15872 raft_consensus.cc:686] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 LEADER]: Becoming Leader. State: Replica: 12e54dd8327a4d48a27207bc73dbfd08, State: Running, Role: LEADER
I20260806 16:17:06.935114 15872 consensus_queue.cc:237] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:06.939021 14959 catalog_manager.cc:5495] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12e54dd8327a4d48a27207bc73dbfd08 (127.11.245.130). New cstate: current_term: 1 leader_uuid: "12e54dd8327a4d48a27207bc73dbfd08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } health_report { overall_health: UNKNOWN } } }
I20260806 16:17:06.969414 15890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:06.964851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41324 (local address 127.11.245.130:44595)
0806 16:17:06.965195 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:06.965204 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:06.965219 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:17:06.965326 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:06.965328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:06.965397 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:06.965558 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:06.965912 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:06.966556 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:06.967473 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:06.967749 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:06.967839 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:06.968298 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:06.968979 (+ 681us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:06.969084 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:06.969180 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 16:17:06.969244 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":168,"threads_started":1}
I20260806 16:17:06.984179 15881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:06.980281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:45001 (local address 127.11.245.131:35401)
0806 16:17:06.980453 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:06.980457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:06.980473 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:17:06.980641 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:06.980644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:06.980703 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:06.980782 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:06.981140 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:06.981722 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:06.982732 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:06.983058 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:06.983155 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:06.983540 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:06.983637 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:06.983701 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:06.983879 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 16:17:06.983942 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 16:17:06.984236 15893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:06.979902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:35401 (local address 127.11.245.130:45001)
0806 16:17:06.980349 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:06.980367 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:06.980515 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:06.980860 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:06.980863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:06.980872 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:17:06.981025 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:06.981031 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:06.981868 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:06.981871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:06.982587 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:06.982598 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:06.983187 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:06.983191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:06.983286 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:06.983352 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:17:06.983763 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:17:06.983771 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 16:17:06.983851 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 16:17:06.983941 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":361,"threads_started":1}
I20260806 16:17:06.993302 15587 raft_consensus.cc:1261] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 FOLLOWER]: Refusing update from remote peer 12e54dd8327a4d48a27207bc73dbfd08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:06.993291 15210 raft_consensus.cc:1261] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28 [term 1 FOLLOWER]: Refusing update from remote peer 12e54dd8327a4d48a27207bc73dbfd08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:06.993901 15872 consensus_queue.cc:1035] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:06.994160 15883 consensus_queue.cc:1035] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:07.017277 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:07.013144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:44595 (local address 127.11.245.131:40605)
0806 16:17:07.013561 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:07.013582 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:07.013712 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:07.014150 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:07.014153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:07.014162 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:17:07.014295 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:07.014302 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:07.015122 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:07.015128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:07.015945 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:07.015956 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:07.016473 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:07.016476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:07.016570 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:07.016613 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:17:07.016951 (+ 338us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:17:07.016960 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 16:17:07.017027 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 16:17:07.017130 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":184,"threads_started":1}
I20260806 16:17:07.017277 15890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:07.013801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:40605 (local address 127.11.245.130:44595)
0806 16:17:07.013933 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:07.013937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:07.013948 (+ 11us) server_negotiation.cc:407] Connection header received
0806 16:17:07.013995 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:07.013997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:07.014048 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:07.014139 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:07.014419 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:07.015004 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:07.016096 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:07.016344 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:07.016474 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:07.016751 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:07.016840 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:07.016942 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:07.017038 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 16:17:07.017098 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 16:17:07.044256 14960 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:07.044543 14960 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:07.047452 14960 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d: Sending DeleteTablet for 3 replicas of tablet d5e5ef074b934a379492ee806a7fb5e3
I20260806 16:17:07.048344 15372 tablet_service.cc:1514] Processing DeleteTablet for tablet d5e5ef074b934a379492ee806a7fb5e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:07 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:41322
I20260806 16:17:07.048377 15562 tablet_service.cc:1514] Processing DeleteTablet for tablet d5e5ef074b934a379492ee806a7fb5e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:07 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:54244
I20260806 16:17:07.048604 15844 tablet_replica.cc:330] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08: stopping tablet replica
I20260806 16:17:07.048605 15847 tablet_replica.cc:330] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236: stopping tablet replica
I20260806 16:17:07.048741 15844 raft_consensus.cc:2229] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:17:07.048748 15847 raft_consensus.cc:2229] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:07.048730 15181 tablet_service.cc:1514] Processing DeleteTablet for tablet d5e5ef074b934a379492ee806a7fb5e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:07 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:59132
I20260806 16:17:07.048910 15847 raft_consensus.cc:2258] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:07.049115 15842 tablet_replica.cc:330] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28: stopping tablet replica
I20260806 16:17:07.049180 15844 raft_consensus.cc:2258] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:07.049278 15842 raft_consensus.cc:2229] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:07.049423 15842 raft_consensus.cc:2258] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:07.049883 14960 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33016:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 16:17:07.050094 15844 ts_tablet_manager.cc:1905] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 16:17:07.050460 14960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:17:07.051225 15842 ts_tablet_manager.cc:1905] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:07.054237 15844 ts_tablet_manager.cc:1918] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:17:07.054258 15842 ts_tablet_manager.cc:1918] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:17:07.054342 15844 log.cc:1196] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/d5e5ef074b934a379492ee806a7fb5e3
I20260806 16:17:07.054368 15842 log.cc:1196] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-0/wal/wals/d5e5ef074b934a379492ee806a7fb5e3
I20260806 16:17:07.054781 15844 ts_tablet_manager.cc:1939] T d5e5ef074b934a379492ee806a7fb5e3 P 12e54dd8327a4d48a27207bc73dbfd08: Deleting consensus metadata
I20260806 16:17:07.054997 15842 ts_tablet_manager.cc:1939] T d5e5ef074b934a379492ee806a7fb5e3 P 945ddb389c3f48b6ab135573c825de28: Deleting consensus metadata
I20260806 16:17:07.055824 14936 catalog_manager.cc:4841] TS 12e54dd8327a4d48a27207bc73dbfd08 (127.11.245.130:44595): tablet d5e5ef074b934a379492ee806a7fb5e3 (table test-table [id=91819665bde74bb48a6cb1d96d583682]) successfully deleted
I20260806 16:17:07.056655 15847 ts_tablet_manager.cc:1905] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:07.055824 14934 catalog_manager.cc:4841] TS 945ddb389c3f48b6ab135573c825de28 (127.11.245.129:39757): tablet d5e5ef074b934a379492ee806a7fb5e3 (table test-table [id=91819665bde74bb48a6cb1d96d583682]) successfully deleted
I20260806 16:17:07.059507 15181 tablet_service.cc:1467] Processing CreateTablet for tablet 77fce3cb404547be8db6e68a2edaa08b (DEFAULT_TABLE table=test-table [id=cd1c8611d3e9491c9854c435d0bd7667]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:07.059698 15372 tablet_service.cc:1467] Processing CreateTablet for tablet 77fce3cb404547be8db6e68a2edaa08b (DEFAULT_TABLE table=test-table [id=cd1c8611d3e9491c9854c435d0bd7667]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:07.059911 15181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77fce3cb404547be8db6e68a2edaa08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:07.060039 15372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77fce3cb404547be8db6e68a2edaa08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:07.063040 15860 tablet_bootstrap.cc:492] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08: Bootstrap starting.
I20260806 16:17:07.063408 15862 tablet_bootstrap.cc:492] T 77fce3cb404547be8db6e68a2edaa08b P 945ddb389c3f48b6ab135573c825de28: Bootstrap starting.
I20260806 16:17:07.063706 15562 tablet_service.cc:1467] Processing CreateTablet for tablet 77fce3cb404547be8db6e68a2edaa08b (DEFAULT_TABLE table=test-table [id=cd1c8611d3e9491c9854c435d0bd7667]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:07.064086 15562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77fce3cb404547be8db6e68a2edaa08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:07.064431 15862 tablet_bootstrap.cc:654] T 77fce3cb404547be8db6e68a2edaa08b P 945ddb389c3f48b6ab135573c825de28: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:07.064435 15860 tablet_bootstrap.cc:654] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:07.066715 15847 ts_tablet_manager.cc:1918] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:17:07.067003 15847 log.cc:1196] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/d5e5ef074b934a379492ee806a7fb5e3
I20260806 16:17:07.067092 15860 tablet_bootstrap.cc:492] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08: No bootstrap required, opened a new log
I20260806 16:17:07.067227 15860 ts_tablet_manager.cc:1397] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:17:07.067487 15847 ts_tablet_manager.cc:1939] T d5e5ef074b934a379492ee806a7fb5e3 P 349bce4fe2e7407bb3ad0407b5a8e236: Deleting consensus metadata
I20260806 16:17:07.067528 15861 tablet_bootstrap.cc:492] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236: Bootstrap starting.
I20260806 16:17:07.067857 15860 raft_consensus.cc:348] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:07.068045 15860 raft_consensus.cc:374] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:07.068135 15860 raft_consensus.cc:729] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e54dd8327a4d48a27207bc73dbfd08, State: Initialized, Role: FOLLOWER
I20260806 16:17:07.068315 15860 consensus_queue.cc:260] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:07.068706 15861 tablet_bootstrap.cc:654] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:07.068717 15860 ts_tablet_manager.cc:1428] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260806 16:17:07.069543 15862 tablet_bootstrap.cc:492] T 77fce3cb404547be8db6e68a2edaa08b P 945ddb389c3f48b6ab135573c825de28: No bootstrap required, opened a new log
I20260806 16:17:07.069653 15862 ts_tablet_manager.cc:1397] T 77fce3cb404547be8db6e68a2edaa08b P 945ddb389c3f48b6ab135573c825de28: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 16:17:07.070134 15862 raft_consensus.cc:348] T 77fce3cb404547be8db6e68a2edaa08b P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:07.070258 15862 raft_consensus.cc:374] T 77fce3cb404547be8db6e68a2edaa08b P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:07.070290 15862 raft_consensus.cc:729] T 77fce3cb404547be8db6e68a2edaa08b P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945ddb389c3f48b6ab135573c825de28, State: Initialized, Role: FOLLOWER
I20260806 16:17:07.070423 15862 consensus_queue.cc:260] T 77fce3cb404547be8db6e68a2edaa08b P 945ddb389c3f48b6ab135573c825de28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:07.070706 15862 ts_tablet_manager.cc:1428] T 77fce3cb404547be8db6e68a2edaa08b P 945ddb389c3f48b6ab135573c825de28: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:17:07.070945 14934 catalog_manager.cc:4841] TS 349bce4fe2e7407bb3ad0407b5a8e236 (127.11.245.131:35401): tablet d5e5ef074b934a379492ee806a7fb5e3 (table test-table [id=91819665bde74bb48a6cb1d96d583682]) successfully deleted
I20260806 16:17:07.071265 15861 tablet_bootstrap.cc:492] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236: No bootstrap required, opened a new log
I20260806 16:17:07.071375 15861 ts_tablet_manager.cc:1397] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:17:07.071825 15861 raft_consensus.cc:348] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:07.071971 15861 raft_consensus.cc:374] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:07.072021 15861 raft_consensus.cc:729] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349bce4fe2e7407bb3ad0407b5a8e236, State: Initialized, Role: FOLLOWER
I20260806 16:17:07.072206 15861 consensus_queue.cc:260] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:07.072546 15861 ts_tablet_manager.cc:1428] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260806 16:17:07.144210 15872 raft_consensus.cc:482] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:07.144428 15872 raft_consensus.cc:504] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:07.145042 15872 leader_election.cc:288] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 945ddb389c3f48b6ab135573c825de28 (127.11.245.129:39757), 349bce4fe2e7407bb3ad0407b5a8e236 (127.11.245.131:35401)
I20260806 16:17:07.145642 15210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77fce3cb404547be8db6e68a2edaa08b" candidate_uuid: "12e54dd8327a4d48a27207bc73dbfd08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945ddb389c3f48b6ab135573c825de28" is_pre_election: true
I20260806 16:17:07.145855 15210 raft_consensus.cc:2454] T 77fce3cb404547be8db6e68a2edaa08b P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12e54dd8327a4d48a27207bc73dbfd08 in term 0.
I20260806 16:17:07.145850 15587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77fce3cb404547be8db6e68a2edaa08b" candidate_uuid: "12e54dd8327a4d48a27207bc73dbfd08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" is_pre_election: true
I20260806 16:17:07.146032 15587 raft_consensus.cc:2454] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12e54dd8327a4d48a27207bc73dbfd08 in term 0.
I20260806 16:17:07.146292 15303 leader_election.cc:302] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12e54dd8327a4d48a27207bc73dbfd08, 945ddb389c3f48b6ab135573c825de28; no voters:
I20260806 16:17:07.146584 15872 raft_consensus.cc:2781] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:07.146683 15872 raft_consensus.cc:482] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:07.146744 15872 raft_consensus.cc:3037] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:07.148190 15872 raft_consensus.cc:504] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:07.148586 15872 leader_election.cc:288] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [CANDIDATE]: Term 1 election: Requested vote from peers 945ddb389c3f48b6ab135573c825de28 (127.11.245.129:39757), 349bce4fe2e7407bb3ad0407b5a8e236 (127.11.245.131:35401)
I20260806 16:17:07.149116 15210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77fce3cb404547be8db6e68a2edaa08b" candidate_uuid: "12e54dd8327a4d48a27207bc73dbfd08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945ddb389c3f48b6ab135573c825de28"
I20260806 16:17:07.149115 15587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77fce3cb404547be8db6e68a2edaa08b" candidate_uuid: "12e54dd8327a4d48a27207bc73dbfd08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349bce4fe2e7407bb3ad0407b5a8e236"
I20260806 16:17:07.149289 15587 raft_consensus.cc:3037] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:07.149289 15210 raft_consensus.cc:3037] T 77fce3cb404547be8db6e68a2edaa08b P 945ddb389c3f48b6ab135573c825de28 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:07.150844 15210 raft_consensus.cc:2454] T 77fce3cb404547be8db6e68a2edaa08b P 945ddb389c3f48b6ab135573c825de28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12e54dd8327a4d48a27207bc73dbfd08 in term 1.
I20260806 16:17:07.150844 15587 raft_consensus.cc:2454] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12e54dd8327a4d48a27207bc73dbfd08 in term 1.
I20260806 16:17:07.151257 15303 leader_election.cc:302] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12e54dd8327a4d48a27207bc73dbfd08, 349bce4fe2e7407bb3ad0407b5a8e236; no voters:
I20260806 16:17:07.151477 15872 raft_consensus.cc:2781] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:07.151583 15872 raft_consensus.cc:686] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 LEADER]: Becoming Leader. State: Replica: 12e54dd8327a4d48a27207bc73dbfd08, State: Running, Role: LEADER
I20260806 16:17:07.151754 15872 consensus_queue.cc:237] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } }
I20260806 16:17:07.154302 14960 catalog_manager.cc:5495] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12e54dd8327a4d48a27207bc73dbfd08 (127.11.245.130). New cstate: current_term: 1 leader_uuid: "12e54dd8327a4d48a27207bc73dbfd08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12e54dd8327a4d48a27207bc73dbfd08" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 } health_report { overall_health: UNKNOWN } } }
I20260806 16:17:07.181200 15210 raft_consensus.cc:1261] T 77fce3cb404547be8db6e68a2edaa08b P 945ddb389c3f48b6ab135573c825de28 [term 1 FOLLOWER]: Refusing update from remote peer 12e54dd8327a4d48a27207bc73dbfd08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:07.181200 15587 raft_consensus.cc:1261] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 FOLLOWER]: Refusing update from remote peer 12e54dd8327a4d48a27207bc73dbfd08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:07.181738 15896 consensus_queue.cc:1035] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349bce4fe2e7407bb3ad0407b5a8e236" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:07.181977 15872 consensus_queue.cc:1035] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "945ddb389c3f48b6ab135573c825de28" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 16:17:07.200999 15818 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 16:17:08.765513 14960 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:08.765743 14960 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:08.768836 14960 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88a1e6b33c7f44b2bfacfaf186f52f0d: Sending DeleteTablet for 3 replicas of tablet 77fce3cb404547be8db6e68a2edaa08b
I20260806 16:17:08.769766 15559 tablet_service.cc:1514] Processing DeleteTablet for tablet 77fce3cb404547be8db6e68a2edaa08b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:08 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:54244
I20260806 16:17:08.769765 15372 tablet_service.cc:1514] Processing DeleteTablet for tablet 77fce3cb404547be8db6e68a2edaa08b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:08 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:41322
I20260806 16:17:08.769776 15181 tablet_service.cc:1514] Processing DeleteTablet for tablet 77fce3cb404547be8db6e68a2edaa08b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:08 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:59132
I20260806 16:17:08.770195 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 15077
I20260806 16:17:08.770254 15946 tablet_replica.cc:330] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236: stopping tablet replica
I20260806 16:17:08.775728 15946 raft_consensus.cc:2229] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:08.775959 15946 raft_consensus.cc:2258] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:08.776328 15950 tablet_replica.cc:330] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08: stopping tablet replica
I20260806 16:17:08.776522 15950 raft_consensus.cc:2229] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:17:08.776829 15950 raft_consensus.cc:2258] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:08.777820 15950 ts_tablet_manager.cc:1905] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:08.778385 15946 ts_tablet_manager.cc:1905] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:08.781536 15946 ts_tablet_manager.cc:1918] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 16:17:08.781536 15950 ts_tablet_manager.cc:1918] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 16:17:08.781636 15950 log.cc:1196] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/77fce3cb404547be8db6e68a2edaa08b
I20260806 16:17:08.781636 15946 log.cc:1196] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/77fce3cb404547be8db6e68a2edaa08b
I20260806 16:17:08.781963 15950 ts_tablet_manager.cc:1939] T 77fce3cb404547be8db6e68a2edaa08b P 12e54dd8327a4d48a27207bc73dbfd08: Deleting consensus metadata
I20260806 16:17:08.781965 15946 ts_tablet_manager.cc:1939] T 77fce3cb404547be8db6e68a2edaa08b P 349bce4fe2e7407bb3ad0407b5a8e236: Deleting consensus metadata
I20260806 16:17:08.783013 14934 catalog_manager.cc:4841] TS 349bce4fe2e7407bb3ad0407b5a8e236 (127.11.245.131:35401): tablet 77fce3cb404547be8db6e68a2edaa08b (table test-table [id=cd1c8611d3e9491c9854c435d0bd7667]) successfully deleted
I20260806 16:17:08.783296 14936 catalog_manager.cc:4841] TS 12e54dd8327a4d48a27207bc73dbfd08 (127.11.245.130:44595): tablet 77fce3cb404547be8db6e68a2edaa08b (table test-table [id=cd1c8611d3e9491c9854c435d0bd7667]) successfully deleted
W20260806 16:17:08.787590 14934 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:39757 (error 108)
I20260806 16:17:08.787883 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 15281
I20260806 16:17:08.795212 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:08.791848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:39757 (local address 127.0.0.1:59156)
0806 16:17:08.794784 (+ 2936us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:08.794913 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:39757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2798,"thread_start_us":122,"threads_started":1}
W20260806 16:17:08.795511 14934 catalog_manager.cc:4584] TS 945ddb389c3f48b6ab135573c825de28 (127.11.245.129:39757): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77fce3cb404547be8db6e68a2edaa08b: Network error: Client connection negotiation failed: client connection to 127.11.245.129:39757: connect: Connection refused (error 111)
I20260806 16:17:08.806396 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 15487
I20260806 16:17:08.827334 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 14907
2026-08-06T16:17:08Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7419 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:17:08 dist-test-slave-t3q3 krb5kdc[15957](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:17:08 dist-test-slave-t3q3 krb5kdc[15957](info): set up 2 sockets
Aug 06 16:17:08 dist-test-slave-t3q3 krb5kdc[15957](info): commencing operation
krb5kdc: starting...
W20260806 16:17:10.129513 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.006s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:17:10 dist-test-slave-t3q3 krb5kdc[15957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:17:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:17:10.293005 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:33971
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:33971
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:10.442193 15974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:10.442551 15974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:10.442600 15974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:10.447073 15974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:10.447160 15974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:10.447180 15974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:10.447198 15974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:10.447214 15974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:17:10.453356 15974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:33971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:33971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:10.454860 15974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:10.456207 15974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:17:10.467823 15974 server_base.cc:1034] running on GCE node
W20260806 16:17:10.467753 15982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:10.467773 15987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:10.467753 15979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:10.468998 15974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:10.470417 15974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:10.471633 15974 hybrid_clock.cc:648] HybridClock initialized: now 1786033030471605 us; error 41 us; skew 500 ppm
Aug 06 16:17:10 dist-test-slave-t3q3 krb5kdc[15957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033030, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:10.475250 15974 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:17:10.476577 15974 webserver.cc:460] Webserver started at http://127.11.245.190:34453/ using document root <none> and password file <none>
I20260806 16:17:10.477344 15974 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:10.477458 15974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:10.477764 15974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:10.480605 15974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "3916ff01f5264affbd506b72908f4937"
format_stamp: "Formatted at 2026-08-06 16:17:10 on dist-test-slave-t3q3"
server_key: "a689bb7b01583bb995c64c18182c1b14"
server_key_iv: "b9c977e9ba257d1e80acdcc71dffb21c"
server_key_version: "encryptionkey@0"
I20260806 16:17:10.481527 15974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "3916ff01f5264affbd506b72908f4937"
format_stamp: "Formatted at 2026-08-06 16:17:10 on dist-test-slave-t3q3"
server_key: "a689bb7b01583bb995c64c18182c1b14"
server_key_iv: "b9c977e9ba257d1e80acdcc71dffb21c"
server_key_version: "encryptionkey@0"
I20260806 16:17:10.487210 15974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:17:10.491055 15996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:10.493232 15974 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:17:10.493554 15974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "3916ff01f5264affbd506b72908f4937"
format_stamp: "Formatted at 2026-08-06 16:17:10 on dist-test-slave-t3q3"
server_key: "a689bb7b01583bb995c64c18182c1b14"
server_key_iv: "b9c977e9ba257d1e80acdcc71dffb21c"
server_key_version: "encryptionkey@0"
I20260806 16:17:10.493767 15974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:10.517865 15974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:10.522099 15974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:10.522392 15974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:10.531850 15974 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:33971
I20260806 16:17:10.531881 16080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:33971 every 8 connection(s)
I20260806 16:17:10.533311 15974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:17:10.535347 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 15974
I20260806 16:17:10.535499 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:17:10.535835 12246 external_mini_cluster.cc:1428] Setting key 8ca391512b721193bfec66323206313e
I20260806 16:17:10.537436 16081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:10.545862 16081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937: Bootstrap starting.
Aug 06 16:17:10 dist-test-slave-t3q3 krb5kdc[15957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:10.549691 16081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:10.550915 16081 log.cc:824] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:10.554440 16081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937: No bootstrap required, opened a new log
I20260806 16:17:10.554735 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:10.537743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36260 (local address 127.11.245.190:33971)
0806 16:17:10.538297 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:10.538317 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:10.538372 (+ 55us) server_negotiation.cc:407] Connection header received
0806 16:17:10.539187 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:10.539206 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:10.539645 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:10.540065 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:10.541041 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:10.542370 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:10.544030 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:10.544693 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:10.544924 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:10.548499 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:10.548506 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:10.548526 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:10.548569 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:10.550765 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:10.551413 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:10.551419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:10.551426 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:10.551635 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:10.552001 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:10.552008 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:10.552013 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:10.552639 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:10.552832 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:10.553234 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 16:17:10.553691 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":191,"threads_started":1}
I20260806 16:17:10.558851 16081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3916ff01f5264affbd506b72908f4937" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33971 } }
I20260806 16:17:10.559129 16081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:10.559221 16081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3916ff01f5264affbd506b72908f4937, State: Initialized, Role: FOLLOWER
I20260806 16:17:10.559775 16081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3916ff01f5264affbd506b72908f4937" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33971 } }
I20260806 16:17:10.559937 16081 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:17:10.560031 16081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:17:10.560209 16081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:10.561407 16081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3916ff01f5264affbd506b72908f4937" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33971 } }
I20260806 16:17:10.561825 16081 leader_election.cc:302] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3916ff01f5264affbd506b72908f4937; no voters:
I20260806 16:17:10.562234 16081 leader_election.cc:288] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:17:10.562423 16086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:10.562736 16086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [term 1 LEADER]: Becoming Leader. State: Replica: 3916ff01f5264affbd506b72908f4937, State: Running, Role: LEADER
I20260806 16:17:10.563105 16086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3916ff01f5264affbd506b72908f4937" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33971 } }
I20260806 16:17:10.563956 16081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:10.565461 16092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3916ff01f5264affbd506b72908f4937. Latest consensus state: current_term: 1 leader_uuid: "3916ff01f5264affbd506b72908f4937" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3916ff01f5264affbd506b72908f4937" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33971 } } }
I20260806 16:17:10.565471 16088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3916ff01f5264affbd506b72908f4937" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3916ff01f5264affbd506b72908f4937" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33971 } } }
I20260806 16:17:10.565620 16092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:10.565634 16088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937 [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:10.566039 16101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:10.569934 16101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:10.577145 16101 catalog_manager.cc:1347] Generated new cluster ID: 7aeebde972f243e0b22dd9408db36276
I20260806 16:17:10.577256 16101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:10.597700 16101 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:17:10.599413 16101 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:10.621732 16101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3916ff01f5264affbd506b72908f4937: Generated new TSK 0
I20260806 16:17:10.623015 16101 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:17:10.661516 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33971
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:10.818500 16118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:10.818888 16118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:10.818945 16118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:10.823206 16118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:10.823272 16118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:10.823348 16118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:17:10.829734 16118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:10.831234 16118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:10.832602 16118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:10.845268 16127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:10.845279 16125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:10.845260 16124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:10.845551 16118 server_base.cc:1034] running on GCE node
I20260806 16:17:10.846186 16118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:10.847002 16118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:10.848148 16118 hybrid_clock.cc:648] HybridClock initialized: now 1786033030848133 us; error 27 us; skew 500 ppm
Aug 06 16:17:10 dist-test-slave-t3q3 krb5kdc[15957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033030, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:10.851274 16118 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:17:10.852401 16118 webserver.cc:460] Webserver started at http://127.11.245.129:43683/ using document root <none> and password file <none>
I20260806 16:17:10.853075 16118 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:10.853144 16118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:10.853355 16118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:10.855991 16118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "24da20add24a42aabc090856f9a59828"
format_stamp: "Formatted at 2026-08-06 16:17:10 on dist-test-slave-t3q3"
server_key: "b532c313535c52b4de54eb7d1a066b3a"
server_key_iv: "1dab84aaec732d55d330cb126e7ece43"
server_key_version: "encryptionkey@0"
I20260806 16:17:10.856765 16118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "24da20add24a42aabc090856f9a59828"
format_stamp: "Formatted at 2026-08-06 16:17:10 on dist-test-slave-t3q3"
server_key: "b532c313535c52b4de54eb7d1a066b3a"
server_key_iv: "1dab84aaec732d55d330cb126e7ece43"
server_key_version: "encryptionkey@0"
I20260806 16:17:10.861451 16118 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:17:10.865098 16136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:10.866881 16118 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:17:10.867044 16118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "24da20add24a42aabc090856f9a59828"
format_stamp: "Formatted at 2026-08-06 16:17:10 on dist-test-slave-t3q3"
server_key: "b532c313535c52b4de54eb7d1a066b3a"
server_key_iv: "1dab84aaec732d55d330cb126e7ece43"
server_key_version: "encryptionkey@0"
I20260806 16:17:10.867182 16118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:10.880836 16118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:10.884706 16118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:10.884936 16118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:10.885756 16118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:10.887326 16118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:10.887426 16118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:10.887513 16118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:10.887547 16118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:10.900641 16118 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:36091
I20260806 16:17:10.900682 16303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:36091 every 8 connection(s)
I20260806 16:17:10.902287 16118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:17:10.910533 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 16118
I20260806 16:17:10.910712 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:17:10.911113 12246 external_mini_cluster.cc:1428] Setting key 9f18e9397976789ef47ec157302c4110
Aug 06 16:17:10 dist-test-slave-t3q3 krb5kdc[15957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033030, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:10.919899 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:10.905236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:38623 (local address 127.11.245.190:33971)
0806 16:17:10.905448 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:10.905455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:10.907058 (+ 1603us) server_negotiation.cc:407] Connection header received
0806 16:17:10.908338 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:10.908341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:10.908402 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:10.908483 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:10.909879 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:10.910480 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:10.912149 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:10.912548 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:10.912673 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:10.915905 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:10.915909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:10.915911 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:10.915937 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:10.917600 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:10.918173 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:10.918181 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:10.918183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:10.918264 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:10.918636 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:10.918641 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:10.918642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:10.918852 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:10.918967 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:10.919558 (+ 591us) server_negotiation.cc:299] Negotiation successful
0806 16:17:10.919705 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":74}
I20260806 16:17:10.920996 16306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:10.905822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:33971 (local address 127.11.245.129:38623)
0806 16:17:10.906814 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:10.906876 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:10.908092 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:10.908645 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:10.908652 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:10.909111 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:10.909724 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:10.909740 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:10.910627 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:10.910634 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:10.911987 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:10.911999 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:10.912724 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:10.912728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:10.912957 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:10.913796 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:10.913814 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:10.915704 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:10.917735 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:10.917742 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:10.917760 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:10.918052 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:10.918365 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:10.918368 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:10.918371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:10.918530 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:10.918967 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:10.918976 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:10.919287 (+ 311us) client_negotiation.cc:769] Sending connection context
0806 16:17:10.919509 (+ 222us) client_negotiation.cc:240] Negotiation successful
0806 16:17:10.919893 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":198,"threads_started":1}
I20260806 16:17:10.922497 16304 heartbeater.cc:347] Connected to a master server at 127.11.245.190:33971
I20260806 16:17:10.922817 16304 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:17:10.923487 16304 heartbeater.cc:511] Master 127.11.245.190:33971 requested a full tablet report, sending...
I20260806 16:17:10.925513 16022 ts_manager.cc:194] Registered new tserver with Master: 24da20add24a42aabc090856f9a59828 (127.11.245.129:36091)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
I20260806 16:17:10.927181 16022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:38623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:17:10.947458 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33971
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:11.097047 16309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:11.097378 16309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:11.097461 16309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:11.101941 16309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:11.102008 16309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:11.102114 16309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:17:11.108268 16309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:11.109905 16309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:11.111331 16309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:11.123530 16315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:11.123539 16318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:11.123539 16316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:11.123975 16309 server_base.cc:1034] running on GCE node
I20260806 16:17:11.124693 16309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:11.125548 16309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:11.126772 16309 hybrid_clock.cc:648] HybridClock initialized: now 1786033031126734 us; error 53 us; skew 500 ppm
Aug 06 16:17:11 dist-test-slave-t3q3 krb5kdc[15957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033031, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:11.130139 16309 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:17:11.131290 16309 webserver.cc:460] Webserver started at http://127.11.245.130:35527/ using document root <none> and password file <none>
I20260806 16:17:11.131982 16309 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:11.132050 16309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:11.132380 16309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:11.134940 16309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "a8245fec5fef43d4b355f5a1e7482dab"
format_stamp: "Formatted at 2026-08-06 16:17:11 on dist-test-slave-t3q3"
server_key: "093e925834ef21e7bc0cf475a922327a"
server_key_iv: "cda703b41fc7af41ebf101bdd758fe69"
server_key_version: "encryptionkey@0"
I20260806 16:17:11.135578 16309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "a8245fec5fef43d4b355f5a1e7482dab"
format_stamp: "Formatted at 2026-08-06 16:17:11 on dist-test-slave-t3q3"
server_key: "093e925834ef21e7bc0cf475a922327a"
server_key_iv: "cda703b41fc7af41ebf101bdd758fe69"
server_key_version: "encryptionkey@0"
I20260806 16:17:11.140295 16309 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.002s
I20260806 16:17:11.143913 16327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:11.145638 16309 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:17:11.145804 16309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "a8245fec5fef43d4b355f5a1e7482dab"
format_stamp: "Formatted at 2026-08-06 16:17:11 on dist-test-slave-t3q3"
server_key: "093e925834ef21e7bc0cf475a922327a"
server_key_iv: "cda703b41fc7af41ebf101bdd758fe69"
server_key_version: "encryptionkey@0"
I20260806 16:17:11.145972 16309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:11.181229 16309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:11.185415 16309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:11.185699 16309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:11.186525 16309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:11.187664 16309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:11.187742 16309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:11.187826 16309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:11.187877 16309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:11.199625 16309 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:45703
I20260806 16:17:11.199651 16501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:45703 every 8 connection(s)
I20260806 16:17:11.200932 16309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:17:11.208690 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 16309
I20260806 16:17:11.208808 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:17:11.209163 12246 external_mini_cluster.cc:1428] Setting key 2314b8721ec50bcd9626de5f83081850
Aug 06 16:17:11 dist-test-slave-t3q3 krb5kdc[15957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033031, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:11.218385 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:11.203441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:33353 (local address 127.11.245.190:33971)
0806 16:17:11.203652 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:11.203657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:11.205005 (+ 1348us) server_negotiation.cc:407] Connection header received
0806 16:17:11.206113 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:11.206117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:11.206193 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:11.206314 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:11.207790 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:11.208465 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:11.210015 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:11.210336 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:11.210496 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:11.213726 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:11.213730 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:11.213733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:11.213763 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:11.215440 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:11.216251 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:11.216255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:11.216256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:11.216378 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:11.216836 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:11.216838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:11.216840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:11.217074 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:11.217159 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:11.218025 (+ 866us) server_negotiation.cc:299] Negotiation successful
0806 16:17:11.218224 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":82}
I20260806 16:17:11.219726 16504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:11.203937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:33971 (local address 127.11.245.130:33353)
0806 16:17:11.204806 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:11.204857 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:11.205865 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:11.206545 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:11.206553 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:11.207002 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:11.207612 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:11.207630 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:11.208617 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:11.208621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:11.209851 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:11.209867 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:11.210549 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:11.210552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:11.210784 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:11.211624 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:11.211646 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:11.213501 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:11.215602 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:11.215613 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:11.215643 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:11.216114 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:11.216486 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:11.216492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:11.216496 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:11.216721 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:11.217193 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:11.217203 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:11.217685 (+ 482us) client_negotiation.cc:769] Sending connection context
0806 16:17:11.217972 (+ 287us) client_negotiation.cc:240] Negotiation successful
0806 16:17:11.218501 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":139,"threads_started":1}
I20260806 16:17:11.221472 16502 heartbeater.cc:347] Connected to a master server at 127.11.245.190:33971
I20260806 16:17:11.221834 16502 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:11.222559 16502 heartbeater.cc:511] Master 127.11.245.190:33971 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
I20260806 16:17:11.224153 16022 ts_manager.cc:194] Registered new tserver with Master: a8245fec5fef43d4b355f5a1e7482dab (127.11.245.130:45703)
I20260806 16:17:11.224915 16022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:33353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:17:11.249181 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33971
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:11.397576 16512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:11.397985 16512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:11.398067 16512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:11.402673 16512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:11.402788 16512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:11.402894 16512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:17:11.409041 16512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:11.410728 16512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:11.412161 16512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:11.424574 16522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:11.424783 16512 server_base.cc:1034] running on GCE node
W20260806 16:17:11.424580 16518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:11.424563 16520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:11.425480 16512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:11.426294 16512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:11.427460 16512 hybrid_clock.cc:648] HybridClock initialized: now 1786033031427425 us; error 52 us; skew 500 ppm
Aug 06 16:17:11 dist-test-slave-t3q3 krb5kdc[15957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033031, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:11.431048 16512 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:17:11.432291 16512 webserver.cc:460] Webserver started at http://127.11.245.131:41149/ using document root <none> and password file <none>
I20260806 16:17:11.433043 16512 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:11.433109 16512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:11.433382 16512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:11.435907 16512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "424d4f5f0a9e4dfaaec9db79b0fa02c7"
format_stamp: "Formatted at 2026-08-06 16:17:11 on dist-test-slave-t3q3"
server_key: "8a60061a9f2822b28778619df5b8c183"
server_key_iv: "e2ba7066d153803779355c9f880957c3"
server_key_version: "encryptionkey@0"
I20260806 16:17:11.436700 16512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "424d4f5f0a9e4dfaaec9db79b0fa02c7"
format_stamp: "Formatted at 2026-08-06 16:17:11 on dist-test-slave-t3q3"
server_key: "8a60061a9f2822b28778619df5b8c183"
server_key_iv: "e2ba7066d153803779355c9f880957c3"
server_key_version: "encryptionkey@0"
I20260806 16:17:11.441496 16512 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:17:11.444778 16531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:11.446413 16512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:17:11.446583 16512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "424d4f5f0a9e4dfaaec9db79b0fa02c7"
format_stamp: "Formatted at 2026-08-06 16:17:11 on dist-test-slave-t3q3"
server_key: "8a60061a9f2822b28778619df5b8c183"
server_key_iv: "e2ba7066d153803779355c9f880957c3"
server_key_version: "encryptionkey@0"
I20260806 16:17:11.446731 16512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:11.459540 16512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:11.463888 16512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:11.464232 16512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:11.465200 16512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:11.466418 16512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:11.466495 16512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:11.466579 16512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:11.466622 16512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:11.478039 16512 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:34963
I20260806 16:17:11.478063 16694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:34963 every 8 connection(s)
I20260806 16:17:11.479249 16512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:17:11.480721 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 16512
I20260806 16:17:11.480883 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:17:11.481258 12246 external_mini_cluster.cc:1428] Setting key a04a2c30b5020898ad524bb7df92eba9
Aug 06 16:17:11 dist-test-slave-t3q3 krb5kdc[15957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033031, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:11.495879 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:11.481773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:35883 (local address 127.11.245.190:33971)
0806 16:17:11.481951 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:11.481956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:11.483468 (+ 1512us) server_negotiation.cc:407] Connection header received
0806 16:17:11.484579 (+ 1111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:11.484582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:11.484638 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:11.484709 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:11.486155 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:11.486822 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:11.488170 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:11.488437 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:11.488529 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:11.491570 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:11.491574 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:11.491576 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:11.491606 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:11.493142 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:11.493688 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:11.493692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:11.493693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:11.493874 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:11.494260 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:11.494268 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:11.494295 (+ 27us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:11.494605 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:11.494736 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:11.495537 (+ 801us) server_negotiation.cc:299] Negotiation successful
0806 16:17:11.495719 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 16:17:11.497244 16698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:11.482330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:33971 (local address 127.11.245.131:35883)
0806 16:17:11.483252 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:11.483351 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:11.484365 (+ 1014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:11.484919 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:11.484927 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:11.485384 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:11.486000 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:11.486016 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:11.486970 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:11.486973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:11.487981 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:11.487993 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:11.488623 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:11.488626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:11.488810 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:11.489590 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:11.489609 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:11.491370 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:11.493256 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:11.493263 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:11.493280 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:11.493569 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:11.493993 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:11.493995 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:11.493997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:11.494148 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:11.494749 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:11.494756 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:11.495191 (+ 435us) client_negotiation.cc:769] Sending connection context
0806 16:17:11.495447 (+ 256us) client_negotiation.cc:240] Negotiation successful
0806 16:17:11.495891 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":172,"threads_started":1}
I20260806 16:17:11.499444 16696 heartbeater.cc:347] Connected to a master server at 127.11.245.190:33971
I20260806 16:17:11.499830 16696 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:11.500665 16696 heartbeater.cc:511] Master 127.11.245.190:33971 requested a full tablet report, sending...
I20260806 16:17:11.502182 16022 ts_manager.cc:194] Registered new tserver with Master: 424d4f5f0a9e4dfaaec9db79b0fa02c7 (127.11.245.131:34963)
I20260806 16:17:11.502898 16022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:35883
I20260806 16:17:11.508625 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 16:17:11.528510 16084 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:11.526483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36266 (local address 127.11.245.190:33971)
0806 16:17:11.526763 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:11.526769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:11.526796 (+ 27us) server_negotiation.cc:407] Connection header received
0806 16:17:11.526977 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:11.526981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:11.527051 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:11.527153 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:11.528328 (+ 1175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36266: BlockingRecv error: recv got EOF from 127.0.0.1:36266 (error 108)
Metrics: {"server-negotiator.queue_time_us":164}
I20260806 16:17:11.530167 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 16118
I20260806 16:17:11.553155 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 16309
I20260806 16:17:11.570637 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 16512
I20260806 16:17:11.592271 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 15974
2026-08-06T16:17:11Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2761 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:17:11 dist-test-slave-t3q3 krb5kdc[16714](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:17:11 dist-test-slave-t3q3 krb5kdc[16714](info): set up 2 sockets
Aug 06 16:17:11 dist-test-slave-t3q3 krb5kdc[16714](info): commencing operation
krb5kdc: starting...
W20260806 16:17:12.885844 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:17:12 dist-test-slave-t3q3 krb5kdc[16714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:17:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:17:13.037614 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:45493
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:45493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:13.179986 16730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:13.180403 16730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:13.180459 16730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:13.184953 16730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:13.185066 16730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:13.185087 16730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:13.185107 16730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:13.185125 16730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:17:13.191444 16730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:45493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:45493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:13.193053 16730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:13.194458 16730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:13.204872 16737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:13.204910 16736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:13.204869 16742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:13.205356 16730 server_base.cc:1034] running on GCE node
I20260806 16:17:13.206051 16730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:13.207381 16730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:13.208575 16730 hybrid_clock.cc:648] HybridClock initialized: now 1786033033208542 us; error 44 us; skew 500 ppm
Aug 06 16:17:13 dist-test-slave-t3q3 krb5kdc[16714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033033, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:13.212234 16730 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:17:13.213583 16730 webserver.cc:460] Webserver started at http://127.11.245.190:41335/ using document root <none> and password file <none>
I20260806 16:17:13.214323 16730 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:13.214425 16730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:13.214709 16730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:13.217432 16730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "2a3fb1e4650b408bba20ba181ddeab0a"
format_stamp: "Formatted at 2026-08-06 16:17:13 on dist-test-slave-t3q3"
server_key: "728e891dd1d3ce33e768ecae30e21436"
server_key_iv: "b1c436af9f627bea1a52fad351593221"
server_key_version: "encryptionkey@0"
I20260806 16:17:13.218111 16730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "2a3fb1e4650b408bba20ba181ddeab0a"
format_stamp: "Formatted at 2026-08-06 16:17:13 on dist-test-slave-t3q3"
server_key: "728e891dd1d3ce33e768ecae30e21436"
server_key_iv: "b1c436af9f627bea1a52fad351593221"
server_key_version: "encryptionkey@0"
I20260806 16:17:13.222800 16730 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:17:13.226013 16751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:13.227768 16730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 16:17:13.227974 16730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "2a3fb1e4650b408bba20ba181ddeab0a"
format_stamp: "Formatted at 2026-08-06 16:17:13 on dist-test-slave-t3q3"
server_key: "728e891dd1d3ce33e768ecae30e21436"
server_key_iv: "b1c436af9f627bea1a52fad351593221"
server_key_version: "encryptionkey@0"
I20260806 16:17:13.228211 16730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:13.246731 16730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:13.250577 16730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:13.250837 16730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:13.259948 16730 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:45493
I20260806 16:17:13.260299 16822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:45493 every 8 connection(s)
I20260806 16:17:13.261296 16730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:17:13.265332 16824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:13.267699 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 16730
I20260806 16:17:13.267917 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:17:13.268342 12246 external_mini_cluster.cc:1428] Setting key 58a4a337fbf9e419cd42c6841ac83e1c
I20260806 16:17:13.274102 16824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a: Bootstrap starting.
I20260806 16:17:13.277477 16824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:13.278396 16824 log.cc:824] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:17:13 dist-test-slave-t3q3 krb5kdc[16714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:13.281240 16824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a: No bootstrap required, opened a new log
I20260806 16:17:13.286170 16828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:13.270127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41686 (local address 127.11.245.190:45493)
0806 16:17:13.270716 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:13.270737 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:13.270785 (+ 48us) server_negotiation.cc:407] Connection header received
0806 16:17:13.271605 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:13.271632 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:13.272176 (+ 544us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:13.272552 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:13.273680 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:13.274902 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:13.276204 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:13.276800 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:13.276950 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:13.280117 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:13.280124 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:13.280143 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:13.280174 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:13.282458 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:13.282923 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:13.282930 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:13.282938 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:13.283092 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:13.283359 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:13.283362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:13.283364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:13.284018 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:13.284303 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:13.284623 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 16:17:13.284984 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":194,"threads_started":1}
I20260806 16:17:13.286737 16824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3fb1e4650b408bba20ba181ddeab0a" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45493 } }
I20260806 16:17:13.287067 16824 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:13.287122 16824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a3fb1e4650b408bba20ba181ddeab0a, State: Initialized, Role: FOLLOWER
I20260806 16:17:13.287648 16824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3fb1e4650b408bba20ba181ddeab0a" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45493 } }
I20260806 16:17:13.287868 16824 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:17:13.287952 16824 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:17:13.288100 16824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:13.289474 16824 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3fb1e4650b408bba20ba181ddeab0a" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45493 } }
I20260806 16:17:13.289978 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:17:13.289934 16824 leader_election.cc:302] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a3fb1e4650b408bba20ba181ddeab0a; no voters:
I20260806 16:17:13.290366 16824 leader_election.cc:288] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:17:13.290498 16830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:13.290776 16830 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [term 1 LEADER]: Becoming Leader. State: Replica: 2a3fb1e4650b408bba20ba181ddeab0a, State: Running, Role: LEADER
I20260806 16:17:13.291184 16830 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3fb1e4650b408bba20ba181ddeab0a" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45493 } }
I20260806 16:17:13.291921 16824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:13.293442 16836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a3fb1e4650b408bba20ba181ddeab0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3fb1e4650b408bba20ba181ddeab0a" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45493 } } }
I20260806 16:17:13.293624 16836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:13.294158 16831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a3fb1e4650b408bba20ba181ddeab0a. Latest consensus state: current_term: 1 leader_uuid: "2a3fb1e4650b408bba20ba181ddeab0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3fb1e4650b408bba20ba181ddeab0a" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45493 } } }
I20260806 16:17:13.294263 16831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:13.295346 16853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:13.298719 16853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:13.305974 16853 catalog_manager.cc:1347] Generated new cluster ID: ed9eb4e4c77947c088c31170e24e6ee0
I20260806 16:17:13.306073 16853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:13.313793 16853 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:17:13.315222 16853 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:13.321211 16828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:13.314188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41692 (local address 127.11.245.190:45493)
0806 16:17:13.314315 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:13.314321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:13.314725 (+ 404us) server_negotiation.cc:407] Connection header received
0806 16:17:13.314888 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:13.314892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:13.315246 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:13.315360 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:13.316205 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:13.317048 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:13.318317 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:13.318745 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:13.318856 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:13.319922 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:13.319926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:13.319929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:17:13.320690 (+ 761us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:13.320730 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:17:13.320752 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:13.320851 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:13.320961 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 16:17:13.321037 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 16:17:13.325901 16853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a3fb1e4650b408bba20ba181ddeab0a: Generated new TSK 0
I20260806 16:17:13.326743 16853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:17:13.338248 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 16730
2026-08-06T16:17:13Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1744 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:17:13 dist-test-slave-t3q3 krb5kdc[16879](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:17:13 dist-test-slave-t3q3 krb5kdc[16879](info): set up 2 sockets
Aug 06 16:17:13 dist-test-slave-t3q3 krb5kdc[16879](info): commencing operation
krb5kdc: starting...
W20260806 16:17:14.646320 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:17:14 dist-test-slave-t3q3 krb5kdc[16879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:17:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:17:14.803785 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:42349
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:42349
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:14.949441 16897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:14.949877 16897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:14.949958 16897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:14.954398 16897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:14.954527 16897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:14.954569 16897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:14.954630 16897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:14.954687 16897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:17:14.961122 16897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:42349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:42349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:14.962713 16897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:14.964130 16897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:14.975153 16903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:14.975150 16906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:14.975173 16902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:14.975550 16897 server_base.cc:1034] running on GCE node
I20260806 16:17:14.976540 16897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:14.978204 16897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:14.979501 16897 hybrid_clock.cc:648] HybridClock initialized: now 1786033034979440 us; error 88 us; skew 500 ppm
Aug 06 16:17:14 dist-test-slave-t3q3 krb5kdc[16879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033034, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:14.988843 16897 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:17:14.990299 16897 webserver.cc:460] Webserver started at http://127.11.245.190:45225/ using document root <none> and password file <none>
I20260806 16:17:14.991124 16897 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:14.991235 16897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:14.991556 16897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:14.994628 16897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "8c5a4f3c41fb40f4adaa0e21e176281c"
format_stamp: "Formatted at 2026-08-06 16:17:14 on dist-test-slave-t3q3"
server_key: "c4f9bfb7341f7e546a30af754d06c739"
server_key_iv: "7bf707738d53028372f51921aeb48ce4"
server_key_version: "encryptionkey@0"
I20260806 16:17:14.995353 16897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "8c5a4f3c41fb40f4adaa0e21e176281c"
format_stamp: "Formatted at 2026-08-06 16:17:14 on dist-test-slave-t3q3"
server_key: "c4f9bfb7341f7e546a30af754d06c739"
server_key_iv: "7bf707738d53028372f51921aeb48ce4"
server_key_version: "encryptionkey@0"
I20260806 16:17:15.000420 16897 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:17:15.003962 16915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:15.005829 16897 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:17:15.006036 16897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "8c5a4f3c41fb40f4adaa0e21e176281c"
format_stamp: "Formatted at 2026-08-06 16:17:14 on dist-test-slave-t3q3"
server_key: "c4f9bfb7341f7e546a30af754d06c739"
server_key_iv: "7bf707738d53028372f51921aeb48ce4"
server_key_version: "encryptionkey@0"
I20260806 16:17:15.006204 16897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:15.019896 16897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:15.024569 16897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:15.024850 16897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:15.033980 16897 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:42349
I20260806 16:17:15.034333 16996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:42349 every 8 connection(s)
I20260806 16:17:15.035355 16897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:17:15.039286 16997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:15.044663 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 16897
I20260806 16:17:15.044839 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:17:15.045186 12246 external_mini_cluster.cc:1428] Setting key eed3959d1e35547e401a855f672ced13
I20260806 16:17:15.047143 16997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c: Bootstrap starting.
I20260806 16:17:15.050248 16997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:15.051440 16997 log.cc:824] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:15.054112 16997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c: No bootstrap required, opened a new log
Aug 06 16:17:15 dist-test-slave-t3q3 krb5kdc[16879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:15.059671 16997 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5a4f3c41fb40f4adaa0e21e176281c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 42349 } }
I20260806 16:17:15.060150 16997 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:15.060210 16997 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c5a4f3c41fb40f4adaa0e21e176281c, State: Initialized, Role: FOLLOWER
I20260806 16:17:15.061023 16997 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5a4f3c41fb40f4adaa0e21e176281c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 42349 } }
I20260806 16:17:15.061234 16997 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:17:15.061317 16997 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:17:15.061445 16997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:15.062969 16997 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5a4f3c41fb40f4adaa0e21e176281c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 42349 } }
I20260806 16:17:15.063519 16997 leader_election.cc:302] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c5a4f3c41fb40f4adaa0e21e176281c; no voters:
I20260806 16:17:15.064054 16997 leader_election.cc:288] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:17:15.064553 17003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:15.065047 17003 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [term 1 LEADER]: Becoming Leader. State: Replica: 8c5a4f3c41fb40f4adaa0e21e176281c, State: Running, Role: LEADER
I20260806 16:17:15.065490 17003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5a4f3c41fb40f4adaa0e21e176281c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 42349 } }
I20260806 16:17:15.066051 16997 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:15.067167 17003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c5a4f3c41fb40f4adaa0e21e176281c. Latest consensus state: current_term: 1 leader_uuid: "8c5a4f3c41fb40f4adaa0e21e176281c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5a4f3c41fb40f4adaa0e21e176281c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 42349 } } }
I20260806 16:17:15.067318 17003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:15.067281 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:15.047145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44992 (local address 127.11.245.190:42349)
0806 16:17:15.047738 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:15.047758 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:15.047803 (+ 45us) server_negotiation.cc:407] Connection header received
0806 16:17:15.048647 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:15.048670 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:15.049114 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:15.049540 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:15.050710 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:15.051925 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:15.052906 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:15.053458 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:15.053655 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:15.057083 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:15.057091 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:15.057117 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:15.057159 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:15.060732 (+ 3573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:15.062535 (+ 1803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:15.062544 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:15.062553 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:15.062779 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:15.063480 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:15.063485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:15.063487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:15.064315 (+ 828us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:15.064631 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:15.065411 (+ 780us) server_negotiation.cc:299] Negotiation successful
0806 16:17:15.065890 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":197,"threads_started":1}
I20260806 16:17:15.067749 17004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c5a4f3c41fb40f4adaa0e21e176281c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5a4f3c41fb40f4adaa0e21e176281c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 42349 } } }
I20260806 16:17:15.067862 17004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:15.067873 17018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:15.073230 17018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:15.081949 17018 catalog_manager.cc:1347] Generated new cluster ID: bee1149b61fd4684902501fd8acb0afe
I20260806 16:17:15.082073 17018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:15.093549 17018 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:17:15.095419 17018 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:15.111282 17018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c5a4f3c41fb40f4adaa0e21e176281c: Generated new TSK 0
I20260806 16:17:15.112478 17018 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:17:15.159427 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:42349
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:15.310298 17035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:15.310690 17035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:15.310765 17035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:15.315202 17035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:15.315331 17035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:15.315438 17035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:17:15.322136 17035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:42349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:15.323937 17035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:15.325419 17035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:15.337930 17042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:15.338155 17035 server_base.cc:1034] running on GCE node
W20260806 16:17:15.337944 17046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:15.337957 17049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:15.338804 17035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:15.339646 17035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:15.340865 17035 hybrid_clock.cc:648] HybridClock initialized: now 1786033035340842 us; error 50 us; skew 500 ppm
Aug 06 16:17:15 dist-test-slave-t3q3 krb5kdc[16879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033035, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:15.344388 17035 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:17:15.345482 17035 webserver.cc:460] Webserver started at http://127.11.245.129:37445/ using document root <none> and password file <none>
I20260806 16:17:15.346218 17035 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:15.346289 17035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:15.346560 17035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:15.349257 17035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "329d5ce0066e4628a26079e377fc0753"
format_stamp: "Formatted at 2026-08-06 16:17:15 on dist-test-slave-t3q3"
server_key: "430fa08352994bfae7f15ef58e919680"
server_key_iv: "7a4e6db26e7ff75e01629379f6a6a5ae"
server_key_version: "encryptionkey@0"
I20260806 16:17:15.349999 17035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "329d5ce0066e4628a26079e377fc0753"
format_stamp: "Formatted at 2026-08-06 16:17:15 on dist-test-slave-t3q3"
server_key: "430fa08352994bfae7f15ef58e919680"
server_key_iv: "7a4e6db26e7ff75e01629379f6a6a5ae"
server_key_version: "encryptionkey@0"
I20260806 16:17:15.354969 17035 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:17:15.358300 17062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:15.359970 17035 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:15.360200 17035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "329d5ce0066e4628a26079e377fc0753"
format_stamp: "Formatted at 2026-08-06 16:17:15 on dist-test-slave-t3q3"
server_key: "430fa08352994bfae7f15ef58e919680"
server_key_iv: "7a4e6db26e7ff75e01629379f6a6a5ae"
server_key_version: "encryptionkey@0"
I20260806 16:17:15.360363 17035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:15.392669 17035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:15.397007 17035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:15.397279 17035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:15.398087 17035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:15.399173 17035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:15.399250 17035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:15.399343 17035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:15.399397 17035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:15.410543 17035 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:35645
I20260806 16:17:15.410569 17245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:35645 every 8 connection(s)
I20260806 16:17:15.411799 17035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:17:15.420982 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 17035
I20260806 16:17:15.421178 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:17:15.421550 12246 external_mini_cluster.cc:1428] Setting key 69258aa978b361d0cddb74dfa4bbbcaa
Aug 06 16:17:15 dist-test-slave-t3q3 krb5kdc[16879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033035, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:15.429853 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:15.414251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:57583 (local address 127.11.245.190:42349)
0806 16:17:15.414477 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:15.414482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:15.415529 (+ 1047us) server_negotiation.cc:407] Connection header received
0806 16:17:15.417469 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:15.417473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:15.417547 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:15.417639 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:15.419161 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:15.419773 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:15.421110 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:15.421451 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:15.421573 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:15.425365 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:15.425370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:15.425372 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:15.425403 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:15.427184 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:15.427859 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:15.427864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:15.427865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:15.427962 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:15.428424 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:15.428428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:15.428430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:15.428670 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:15.428793 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:15.429472 (+ 679us) server_negotiation.cc:299] Negotiation successful
0806 16:17:15.429637 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 16:17:15.430985 17249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:15.414730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:42349 (local address 127.11.245.129:57583)
0806 16:17:15.415334 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:15.415394 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:15.417173 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:15.417867 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:15.417875 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:15.418350 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:15.418975 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:15.418992 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:15.419930 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:15.419941 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:15.420965 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:15.420975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:15.421638 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:15.421642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:15.421849 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:15.422742 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:15.422770 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:15.425132 (+ 2362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:15.427347 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:15.427356 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:15.427378 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:15.427715 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:15.428101 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:15.428105 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:15.428107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:15.428301 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:15.428815 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:15.428825 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:15.429170 (+ 345us) client_negotiation.cc:769] Sending connection context
0806 16:17:15.429413 (+ 243us) client_negotiation.cc:240] Negotiation successful
0806 16:17:15.429868 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":155,"threads_started":1}
I20260806 16:17:15.432778 17247 heartbeater.cc:347] Connected to a master server at 127.11.245.190:42349
I20260806 16:17:15.433133 17247 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:15.433847 17247 heartbeater.cc:511] Master 127.11.245.190:42349 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:17:15.436144 16942 ts_manager.cc:194] Registered new tserver with Master: 329d5ce0066e4628a26079e377fc0753 (127.11.245.129:35645)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
I20260806 16:17:15.437986 16942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:57583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:17:15.456543 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:42349
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:15.606314 17254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:15.606698 17254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:15.606750 17254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:15.611289 17254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:15.611378 17254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:15.611498 17254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:17:15.619225 17254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:42349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:15.620929 17254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:15.622347 17254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:17:15.635896 17254 server_base.cc:1034] running on GCE node
W20260806 16:17:15.635752 17265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:15.635802 17266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:15.635766 17268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:15.636817 17254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:15.637611 17254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:15.638840 17254 hybrid_clock.cc:648] HybridClock initialized: now 1786033035638819 us; error 37 us; skew 500 ppm
Aug 06 16:17:15 dist-test-slave-t3q3 krb5kdc[16879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033035, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:15.642377 17254 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:17:15.643533 17254 webserver.cc:460] Webserver started at http://127.11.245.130:39031/ using document root <none> and password file <none>
I20260806 16:17:15.644354 17254 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:15.644454 17254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:15.644745 17254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:15.647406 17254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "b3d8645408ee446794087394f2a81ac2"
format_stamp: "Formatted at 2026-08-06 16:17:15 on dist-test-slave-t3q3"
server_key: "b6f0a125904a30495fba593a3d27c04b"
server_key_iv: "3e3c9ab242ce3167959aa8f40166d04b"
server_key_version: "encryptionkey@0"
I20260806 16:17:15.648154 17254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "b3d8645408ee446794087394f2a81ac2"
format_stamp: "Formatted at 2026-08-06 16:17:15 on dist-test-slave-t3q3"
server_key: "b6f0a125904a30495fba593a3d27c04b"
server_key_iv: "3e3c9ab242ce3167959aa8f40166d04b"
server_key_version: "encryptionkey@0"
I20260806 16:17:15.652837 17254 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:17:15.655941 17281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:15.657759 17254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:17:15.657930 17254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "b3d8645408ee446794087394f2a81ac2"
format_stamp: "Formatted at 2026-08-06 16:17:15 on dist-test-slave-t3q3"
server_key: "b6f0a125904a30495fba593a3d27c04b"
server_key_iv: "3e3c9ab242ce3167959aa8f40166d04b"
server_key_version: "encryptionkey@0"
I20260806 16:17:15.658068 17254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:15.671128 17254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:15.675541 17254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:15.675808 17254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:15.676687 17254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:15.677850 17254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:15.677932 17254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:15.678011 17254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:15.678062 17254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:15.689141 17254 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:41045
I20260806 16:17:15.689147 17459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:41045 every 8 connection(s)
I20260806 16:17:15.690366 17254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:17:15.697575 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 17254
I20260806 16:17:15.697714 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:17:15.698100 12246 external_mini_cluster.cc:1428] Setting key 9cda8b0fba601a6375907310170dea61
Aug 06 16:17:15 dist-test-slave-t3q3 krb5kdc[16879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033035, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:15.710335 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:15.692943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:35431 (local address 127.11.245.190:42349)
0806 16:17:15.693159 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:15.693163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:15.694302 (+ 1139us) server_negotiation.cc:407] Connection header received
0806 16:17:15.696227 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:15.696231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:15.696296 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:15.696409 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:15.698122 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:15.698868 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:15.700310 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:15.700563 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:15.700663 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:15.704666 (+ 4003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:15.704672 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:15.704678 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:15.704720 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:15.706668 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:15.707680 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:15.707687 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:15.707692 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:15.707829 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:15.708432 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:15.708436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:15.708437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:15.708725 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:15.708823 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:15.710007 (+ 1184us) server_negotiation.cc:299] Negotiation successful
0806 16:17:15.710171 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260806 16:17:15.712167 17466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:15.693437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:42349 (local address 127.11.245.130:35431)
0806 16:17:15.694089 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:15.694146 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:15.695896 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:15.696672 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:15.696684 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:15.697142 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:15.697940 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:15.697961 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:15.699017 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:15.699028 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:15.700176 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:15.700189 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:15.700764 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:15.700770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:15.701035 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:15.701916 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:15.701946 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:15.704432 (+ 2486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:15.706841 (+ 2409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:15.706854 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:15.706885 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:15.707467 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:15.707958 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:15.707965 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:15.707969 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:15.708273 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:15.708880 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:15.708894 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:15.709530 (+ 636us) client_negotiation.cc:769] Sending connection context
0806 16:17:15.709882 (+ 352us) client_negotiation.cc:240] Negotiation successful
0806 16:17:15.710472 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":154,"threads_started":1}
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
I20260806 16:17:15.714025 17461 heartbeater.cc:347] Connected to a master server at 127.11.245.190:42349
I20260806 16:17:15.714406 17461 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:15.715161 17461 heartbeater.cc:511] Master 127.11.245.190:42349 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
I20260806 16:17:15.716671 16942 ts_manager.cc:194] Registered new tserver with Master: b3d8645408ee446794087394f2a81ac2 (127.11.245.130:41045)
I20260806 16:17:15.717453 16942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:35431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:17:15.736698 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:42349
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:15.883474 17473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:15.883891 17473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:15.883946 17473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:15.888692 17473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:15.888793 17473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:15.888911 17473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:17:15.895539 17473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:42349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:15.897190 17473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:15.898586 17473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:15.911067 17486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:15.911080 17485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:15.911113 17490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:15.911494 17473 server_base.cc:1034] running on GCE node
I20260806 16:17:15.912113 17473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:15.913064 17473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:15.914275 17473 hybrid_clock.cc:648] HybridClock initialized: now 1786033035914257 us; error 34 us; skew 500 ppm
Aug 06 16:17:15 dist-test-slave-t3q3 krb5kdc[16879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033035, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:15.917912 17473 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:17:15.919149 17473 webserver.cc:460] Webserver started at http://127.11.245.131:34455/ using document root <none> and password file <none>
I20260806 16:17:15.919883 17473 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:15.919956 17473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:15.920280 17473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:15.923007 17473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "8a820a54b8ac4adb9ca4f007ad855cc5"
format_stamp: "Formatted at 2026-08-06 16:17:15 on dist-test-slave-t3q3"
server_key: "4d6e8a93e699498d97576208a445e1cc"
server_key_iv: "6c3398e19657292164aa0b109ccf05b3"
server_key_version: "encryptionkey@0"
I20260806 16:17:15.923914 17473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "8a820a54b8ac4adb9ca4f007ad855cc5"
format_stamp: "Formatted at 2026-08-06 16:17:15 on dist-test-slave-t3q3"
server_key: "4d6e8a93e699498d97576208a445e1cc"
server_key_iv: "6c3398e19657292164aa0b109ccf05b3"
server_key_version: "encryptionkey@0"
I20260806 16:17:15.929132 17473 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 16:17:15.932799 17501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:15.934772 17473 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:17:15.934978 17473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "8a820a54b8ac4adb9ca4f007ad855cc5"
format_stamp: "Formatted at 2026-08-06 16:17:15 on dist-test-slave-t3q3"
server_key: "4d6e8a93e699498d97576208a445e1cc"
server_key_iv: "6c3398e19657292164aa0b109ccf05b3"
server_key_version: "encryptionkey@0"
I20260806 16:17:15.935150 17473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:15.975206 17473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:15.979640 17473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:15.979923 17473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:15.980808 17473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:15.981914 17473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:15.981968 17473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:15.982053 17473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:15.982085 17473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:15.993388 17473 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:43031
I20260806 16:17:15.993423 17666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:43031 every 8 connection(s)
I20260806 16:17:15.994553 17473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:17:15.997740 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 17473
I20260806 16:17:15.997905 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:17:15.998351 12246 external_mini_cluster.cc:1428] Setting key 6744a0b9ccb363a7bd7d48228e6fcbe6
Aug 06 16:17:16 dist-test-slave-t3q3 krb5kdc[16879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033035, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:16.011662 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:15.996978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:49399 (local address 127.11.245.190:42349)
0806 16:17:15.997167 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:15.997171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:15.998305 (+ 1134us) server_negotiation.cc:407] Connection header received
0806 16:17:16.000358 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:16.000361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:16.000416 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:16.000492 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:16.002081 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:16.002674 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:16.003904 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:16.004214 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:16.004316 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:16.007282 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:16.007286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:16.007288 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:16.007342 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:16.009130 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:16.009743 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:16.009746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:16.009747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:16.009837 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:16.010235 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:16.010239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:16.010242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:16.010529 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:16.010633 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:16.011258 (+ 625us) server_negotiation.cc:299] Negotiation successful
0806 16:17:16.011465 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 16:17:16.012955 17670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:15.997437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:42349 (local address 127.11.245.131:49399)
0806 16:17:15.998100 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:15.998172 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:16.000013 (+ 1841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:16.000711 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:16.000722 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:16.001273 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:16.001925 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:16.001941 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:16.002792 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:16.002795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:16.003748 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:16.003758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:16.004401 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:16.004405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:16.004604 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:16.005297 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:16.005317 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:16.007069 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:16.009258 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:16.009265 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:16.009284 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:16.009637 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:16.009929 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:16.009931 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:16.009932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:16.010088 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:16.010670 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:16.010677 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:16.010998 (+ 321us) client_negotiation.cc:769] Sending connection context
0806 16:17:16.011189 (+ 191us) client_negotiation.cc:240] Negotiation successful
0806 16:17:16.011578 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":146,"threads_started":1}
I20260806 16:17:16.014339 17668 heartbeater.cc:347] Connected to a master server at 127.11.245.190:42349
I20260806 16:17:16.014690 17668 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:16.015348 17668 heartbeater.cc:511] Master 127.11.245.190:42349 requested a full tablet report, sending...
I20260806 16:17:16.016656 16942 ts_manager.cc:194] Registered new tserver with Master: 8a820a54b8ac4adb9ca4f007ad855cc5 (127.11.245.131:43031)
I20260806 16:17:16.017310 16942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:49399
I20260806 16:17:16.026000 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 16:17:16 dist-test-slave-t3q3 krb5kdc[16879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033036, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 16:17:16 dist-test-slave-t3q3 krb5kdc[16879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033036, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:16.059252 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:16.049539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45000 (local address 127.11.245.190:42349)
0806 16:17:16.049701 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:16.049705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:16.049902 (+ 197us) server_negotiation.cc:407] Connection header received
0806 16:17:16.050082 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:16.050085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:16.050142 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:16.050240 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:16.051064 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:16.051668 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:16.052689 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:16.052923 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:16.053036 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:16.055731 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:16.055735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:16.055737 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:16.055766 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:16.057265 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:16.057725 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:16.057728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:16.057729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:16.057812 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:16.058240 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:16.058243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:16.058245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:16.058546 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:16.058684 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:16.058970 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 16:17:16.059098 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 16:17:16.059882 16942 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45000
I20260806 16:17:16.061151 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 17035
I20260806 16:17:16.078276 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 17254
I20260806 16:17:16.102595 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 17473
I20260806 16:17:16.118620 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 16897
2026-08-06T16:17:16Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2776 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:17:16 dist-test-slave-t3q3 krb5kdc[17684](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:17:16 dist-test-slave-t3q3 krb5kdc[17684](info): set up 2 sockets
Aug 06 16:17:16 dist-test-slave-t3q3 krb5kdc[17684](info): commencing operation
krb5kdc: starting...
W20260806 16:17:17.454474 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:17:17 dist-test-slave-t3q3 krb5kdc[17684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:17:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:17:17.616709 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:40697
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:40697
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:17.763688 17703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:17.764230 17703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:17.764333 17703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:17.769161 17703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:17.769277 17703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:17.769322 17703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:17.769366 17703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:17.769405 17703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:17:17.776818 17703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:40697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:40697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:17.778504 17703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:17.779991 17703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:17:17.792644 17703 server_base.cc:1034] running on GCE node
W20260806 16:17:17.792604 17710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:17.792740 17715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:17.792598 17711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:17.793732 17703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:17.795282 17703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:17.796496 17703 hybrid_clock.cc:648] HybridClock initialized: now 1786033037796476 us; error 34 us; skew 500 ppm
Aug 06 16:17:17 dist-test-slave-t3q3 krb5kdc[17684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033037, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:17.800213 17703 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:17:17.801512 17703 webserver.cc:460] Webserver started at http://127.11.245.190:42165/ using document root <none> and password file <none>
I20260806 16:17:17.802294 17703 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:17.802407 17703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:17.802731 17703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:17.805598 17703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "0c8fa49d58254051b578aa8179968d1d"
format_stamp: "Formatted at 2026-08-06 16:17:17 on dist-test-slave-t3q3"
server_key: "3cd94e192275c8d6cee1dbaff755cb77"
server_key_iv: "d2d73f7da86a4620fe6d7c197629cd0a"
server_key_version: "encryptionkey@0"
I20260806 16:17:17.806388 17703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "0c8fa49d58254051b578aa8179968d1d"
format_stamp: "Formatted at 2026-08-06 16:17:17 on dist-test-slave-t3q3"
server_key: "3cd94e192275c8d6cee1dbaff755cb77"
server_key_iv: "d2d73f7da86a4620fe6d7c197629cd0a"
server_key_version: "encryptionkey@0"
I20260806 16:17:17.811520 17703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:17:17.814873 17725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:17.816691 17703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:17:17.816861 17703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "0c8fa49d58254051b578aa8179968d1d"
format_stamp: "Formatted at 2026-08-06 16:17:17 on dist-test-slave-t3q3"
server_key: "3cd94e192275c8d6cee1dbaff755cb77"
server_key_iv: "d2d73f7da86a4620fe6d7c197629cd0a"
server_key_version: "encryptionkey@0"
I20260806 16:17:17.817013 17703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:17.857360 17703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:17.861462 17703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:17.861733 17703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:17.871877 17703 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:40697
I20260806 16:17:17.871958 17798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:40697 every 8 connection(s)
I20260806 16:17:17.873453 17703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:17:17.877707 17799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:17.878027 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 17703
I20260806 16:17:17.878185 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:17:17.878600 12246 external_mini_cluster.cc:1428] Setting key 16f36433085fe2fce4cbf185dd7fe15d
I20260806 16:17:17.887465 17799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d: Bootstrap starting.
Aug 06 16:17:17 dist-test-slave-t3q3 krb5kdc[17684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:17.891328 17799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:17.892652 17799 log.cc:824] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:17.896179 17799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d: No bootstrap required, opened a new log
I20260806 16:17:17.897589 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:17.880612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46140 (local address 127.11.245.190:40697)
0806 16:17:17.881177 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:17.881198 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:17.881246 (+ 48us) server_negotiation.cc:407] Connection header received
0806 16:17:17.882135 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:17.882160 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:17.882605 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:17.883119 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:17.884487 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:17.885638 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:17.886892 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:17.887467 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:17.887660 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:17.891055 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:17.891066 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:17.891092 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:17.891138 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:17.893491 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:17.894138 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:17.894147 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:17.894156 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:17.894328 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:17.894669 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:17.894673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:17.894675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:17.895356 (+ 681us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:17.895587 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:17.895904 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 16:17:17.896346 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":181,"threads_started":1}
I20260806 16:17:17.901782 17799 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8fa49d58254051b578aa8179968d1d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 40697 } }
I20260806 16:17:17.902120 17799 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:17.902210 17799 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c8fa49d58254051b578aa8179968d1d, State: Initialized, Role: FOLLOWER
I20260806 16:17:17.902716 17799 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8fa49d58254051b578aa8179968d1d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 40697 } }
I20260806 16:17:17.902871 17799 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:17:17.902922 17799 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:17:17.903005 17799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:17.904321 17799 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8fa49d58254051b578aa8179968d1d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 40697 } }
I20260806 16:17:17.904728 17799 leader_election.cc:302] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c8fa49d58254051b578aa8179968d1d; no voters:
I20260806 16:17:17.905120 17799 leader_election.cc:288] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:17:17.905294 17806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:17.905582 17806 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [term 1 LEADER]: Becoming Leader. State: Replica: 0c8fa49d58254051b578aa8179968d1d, State: Running, Role: LEADER
I20260806 16:17:17.906067 17806 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8fa49d58254051b578aa8179968d1d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 40697 } }
I20260806 16:17:17.906975 17799 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:17.908363 17807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c8fa49d58254051b578aa8179968d1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8fa49d58254051b578aa8179968d1d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 40697 } } }
I20260806 16:17:17.908394 17808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c8fa49d58254051b578aa8179968d1d. Latest consensus state: current_term: 1 leader_uuid: "0c8fa49d58254051b578aa8179968d1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8fa49d58254051b578aa8179968d1d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 40697 } } }
I20260806 16:17:17.908514 17807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:17.908524 17808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:17.909014 17819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:17.913918 17819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:17.923385 17819 catalog_manager.cc:1347] Generated new cluster ID: b024654965144d16a5df14ac71b28b2c
I20260806 16:17:17.923535 17819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:17.936223 17819 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:17:17.937510 17819 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:17.946856 17819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c8fa49d58254051b578aa8179968d1d: Generated new TSK 0
I20260806 16:17:17.947918 17819 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:17:17.984922 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:40697
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:18.124910 17837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:18.125250 17837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:18.125301 17837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:18.129459 17837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:18.129515 17837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:18.129577 17837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:17:18.135408 17837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:40697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:18.136765 17837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:18.138089 17837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:18.150681 17843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:18.150681 17846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:18.150681 17849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:18.151357 17837 server_base.cc:1034] running on GCE node
I20260806 16:17:18.151893 17837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:18.152837 17837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:18.154026 17837 hybrid_clock.cc:648] HybridClock initialized: now 1786033038154005 us; error 41 us; skew 500 ppm
Aug 06 16:17:18 dist-test-slave-t3q3 krb5kdc[17684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033038, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:18.157552 17837 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:17:18.158804 17837 webserver.cc:460] Webserver started at http://127.11.245.129:37727/ using document root <none> and password file <none>
I20260806 16:17:18.159575 17837 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:18.159679 17837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:18.159982 17837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:18.162778 17837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "649b4deac7c94f589fbcf3763b3707bb"
format_stamp: "Formatted at 2026-08-06 16:17:18 on dist-test-slave-t3q3"
server_key: "e6c3922bd0b659e726bbf75b033acd12"
server_key_iv: "1625d56bd2deadc16ec867a226d4c757"
server_key_version: "encryptionkey@0"
I20260806 16:17:18.163552 17837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "649b4deac7c94f589fbcf3763b3707bb"
format_stamp: "Formatted at 2026-08-06 16:17:18 on dist-test-slave-t3q3"
server_key: "e6c3922bd0b659e726bbf75b033acd12"
server_key_iv: "1625d56bd2deadc16ec867a226d4c757"
server_key_version: "encryptionkey@0"
I20260806 16:17:18.169046 17837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:17:18.172873 17858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:18.174888 17837 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:17:18.175082 17837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "649b4deac7c94f589fbcf3763b3707bb"
format_stamp: "Formatted at 2026-08-06 16:17:18 on dist-test-slave-t3q3"
server_key: "e6c3922bd0b659e726bbf75b033acd12"
server_key_iv: "1625d56bd2deadc16ec867a226d4c757"
server_key_version: "encryptionkey@0"
I20260806 16:17:18.175253 17837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:18.204965 17837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:18.209532 17837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:18.209816 17837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:18.210628 17837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:18.211755 17837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:18.211833 17837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:18.211938 17837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:18.211982 17837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:18.224781 17837 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:41575
I20260806 16:17:18.224802 18036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:41575 every 8 connection(s)
I20260806 16:17:18.226099 17837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:17:18.235870 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 17837
I20260806 16:17:18.236094 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:17:18.236562 12246 external_mini_cluster.cc:1428] Setting key cce9b801fa9c73cd0c91dd712910e738
Aug 06 16:17:18 dist-test-slave-t3q3 krb5kdc[17684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033038, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:18.245092 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:18.228939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:37607 (local address 127.11.245.190:40697)
0806 16:17:18.229142 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:18.229149 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:18.230382 (+ 1233us) server_negotiation.cc:407] Connection header received
0806 16:17:18.232200 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:18.232203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:18.232270 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:18.232371 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:18.234054 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:18.234667 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:18.236432 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:18.236767 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:18.236893 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:18.240295 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:18.240299 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:18.240302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:18.240332 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:18.242124 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:18.242855 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:18.242859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:18.242861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:18.242953 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:18.243358 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:18.243362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:18.243364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:18.243608 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:18.243727 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:18.244716 (+ 989us) server_negotiation.cc:299] Negotiation successful
0806 16:17:18.244887 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 16:17:18.246446 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:18.229476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:40697 (local address 127.11.245.129:37607)
0806 16:17:18.230177 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:18.230236 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:18.231847 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:18.232597 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:18.232606 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:18.233199 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:18.233859 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:18.233877 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:18.234811 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:18.234814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:18.236257 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:18.236274 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:18.236996 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:18.237000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:18.237225 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:18.238062 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:18.238083 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:18.240015 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:18.242364 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:18.242372 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:18.242397 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:18.242721 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:18.243063 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:18.243066 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:18.243069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:18.243246 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:18.243729 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:18.243741 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:18.244252 (+ 511us) client_negotiation.cc:769] Sending connection context
0806 16:17:18.244674 (+ 422us) client_negotiation.cc:240] Negotiation successful
0806 16:17:18.245168 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":175,"threads_started":1}
I20260806 16:17:18.248386 18037 heartbeater.cc:347] Connected to a master server at 127.11.245.190:40697
I20260806 16:17:18.248764 18037 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:18.249545 18037 heartbeater.cc:511] Master 127.11.245.190:40697 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:17:18.251895 17746 ts_manager.cc:194] Registered new tserver with Master: 649b4deac7c94f589fbcf3763b3707bb (127.11.245.129:41575)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
I20260806 16:17:18.253724 17746 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:37607
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:17:18.277033 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:40697
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:18.426311 18044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:18.426679 18044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:18.426729 18044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:18.431200 18044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:18.431296 18044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:18.431365 18044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:17:18.437830 18044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:40697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:18.439492 18044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:18.440902 18044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:18.453733 18051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:18.453780 18053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:18.453788 18050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:18.454010 18044 server_base.cc:1034] running on GCE node
I20260806 16:17:18.454625 18044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:18.455559 18044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:18.456787 18044 hybrid_clock.cc:648] HybridClock initialized: now 1786033038456766 us; error 42 us; skew 500 ppm
Aug 06 16:17:18 dist-test-slave-t3q3 krb5kdc[17684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033038, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:18.460728 18044 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:17:18.461972 18044 webserver.cc:460] Webserver started at http://127.11.245.130:34457/ using document root <none> and password file <none>
I20260806 16:17:18.462744 18044 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:18.462862 18044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:18.463162 18044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:18.466065 18044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "83542df724b84053ab3086f813cb9b36"
format_stamp: "Formatted at 2026-08-06 16:17:18 on dist-test-slave-t3q3"
server_key: "3946369656ea7d2f7e49301b79beb918"
server_key_iv: "a4dfff4150ca7c64940544a78af355f3"
server_key_version: "encryptionkey@0"
I20260806 16:17:18.466797 18044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "83542df724b84053ab3086f813cb9b36"
format_stamp: "Formatted at 2026-08-06 16:17:18 on dist-test-slave-t3q3"
server_key: "3946369656ea7d2f7e49301b79beb918"
server_key_iv: "a4dfff4150ca7c64940544a78af355f3"
server_key_version: "encryptionkey@0"
I20260806 16:17:18.471717 18044 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 16:17:18.474978 18064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:18.476735 18044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 16:17:18.476955 18044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "83542df724b84053ab3086f813cb9b36"
format_stamp: "Formatted at 2026-08-06 16:17:18 on dist-test-slave-t3q3"
server_key: "3946369656ea7d2f7e49301b79beb918"
server_key_iv: "a4dfff4150ca7c64940544a78af355f3"
server_key_version: "encryptionkey@0"
I20260806 16:17:18.477113 18044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:18.500666 18044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:18.505103 18044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:18.505380 18044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:18.506237 18044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:18.507501 18044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:18.507579 18044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:18.507661 18044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:18.507712 18044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:18.518759 18044 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:34713
I20260806 16:17:18.518801 18243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:34713 every 8 connection(s)
I20260806 16:17:18.519892 18044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:17:18.528980 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 18044
I20260806 16:17:18.529147 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:17:18.529523 12246 external_mini_cluster.cc:1428] Setting key 136c1cbc7cc0570554631a3153949332
Aug 06 16:17:18 dist-test-slave-t3q3 krb5kdc[17684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033038, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:18.539749 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:18.522451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:49633 (local address 127.11.245.190:40697)
0806 16:17:18.522632 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:18.522636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:18.523764 (+ 1128us) server_negotiation.cc:407] Connection header received
0806 16:17:18.525495 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:18.525499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:18.525559 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:18.525656 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:18.527103 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:18.527713 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:18.529875 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:18.530156 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:18.530312 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:18.534167 (+ 3855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:18.534175 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:18.534182 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:18.534231 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:18.536244 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:18.537155 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:18.537160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:18.537161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:18.537303 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:18.537868 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:18.537872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:18.537873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:18.538158 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:18.538268 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:18.539249 (+ 981us) server_negotiation.cc:299] Negotiation successful
0806 16:17:18.539428 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 16:17:18.540927 18246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:18.522961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:40697 (local address 127.11.245.130:49633)
0806 16:17:18.523576 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:18.523628 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:18.525231 (+ 1603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:18.525850 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:18.525857 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:18.526300 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:18.526932 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:18.526948 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:18.528243 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:18.528246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:18.529722 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:18.529737 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:18.530374 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:18.530379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:18.530601 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:18.531458 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:18.531483 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:18.533889 (+ 2406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:18.536435 (+ 2546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:18.536447 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:18.536480 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:18.536989 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:18.537427 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:18.537433 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:18.537438 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:18.537695 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:18.538305 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:18.538318 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:18.538831 (+ 513us) client_negotiation.cc:769] Sending connection context
0806 16:17:18.539140 (+ 309us) client_negotiation.cc:240] Negotiation successful
0806 16:17:18.539684 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":143,"threads_started":1}
I20260806 16:17:18.542546 18244 heartbeater.cc:347] Connected to a master server at 127.11.245.190:40697
I20260806 16:17:18.542913 18244 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:18.543807 18244 heartbeater.cc:511] Master 127.11.245.190:40697 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
I20260806 16:17:18.545527 17746 ts_manager.cc:194] Registered new tserver with Master: 83542df724b84053ab3086f813cb9b36 (127.11.245.130:34713)
I20260806 16:17:18.546352 17746 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:49633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:17:18.568953 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:40697
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:40697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:18.722618 18251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:18.723006 18251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:18.723057 18251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:18.727608 18251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:18.727747 18251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:18.727824 18251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:17:18.736315 18251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:40697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:18.738514 18251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:18.740568 18251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:18.754031 18267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:18.754038 18263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:18.754038 18264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:18.754283 18251 server_base.cc:1034] running on GCE node
I20260806 16:17:18.755028 18251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:18.756011 18251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:18.757264 18251 hybrid_clock.cc:648] HybridClock initialized: now 1786033038757240 us; error 45 us; skew 500 ppm
Aug 06 16:17:18 dist-test-slave-t3q3 krb5kdc[17684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033038, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:18.761359 18251 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:17:18.762549 18251 webserver.cc:460] Webserver started at http://127.11.245.131:40019/ using document root <none> and password file <none>
I20260806 16:17:18.763312 18251 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:18.763396 18251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:18.763636 18251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:18.766508 18251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "a12bb3c6fdea4b13a2d4fa4eb914996f"
format_stamp: "Formatted at 2026-08-06 16:17:18 on dist-test-slave-t3q3"
server_key: "da3292cd34c0d8b34637e314a631773c"
server_key_iv: "9ac12adbd84b57eea49de46cda6b98e5"
server_key_version: "encryptionkey@0"
I20260806 16:17:18.767346 18251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "a12bb3c6fdea4b13a2d4fa4eb914996f"
format_stamp: "Formatted at 2026-08-06 16:17:18 on dist-test-slave-t3q3"
server_key: "da3292cd34c0d8b34637e314a631773c"
server_key_iv: "9ac12adbd84b57eea49de46cda6b98e5"
server_key_version: "encryptionkey@0"
I20260806 16:17:18.773912 18251 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:17:18.777410 18277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:18.779202 18251 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:17:18.779347 18251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "a12bb3c6fdea4b13a2d4fa4eb914996f"
format_stamp: "Formatted at 2026-08-06 16:17:18 on dist-test-slave-t3q3"
server_key: "da3292cd34c0d8b34637e314a631773c"
server_key_iv: "9ac12adbd84b57eea49de46cda6b98e5"
server_key_version: "encryptionkey@0"
I20260806 16:17:18.779481 18251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:18.804612 18251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:18.809513 18251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:18.809741 18251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:18.810628 18251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:18.811769 18251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:18.811836 18251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:18.811895 18251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:18.811919 18251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:18.823532 18251 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:43483
I20260806 16:17:18.823567 18456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:43483 every 8 connection(s)
I20260806 16:17:18.824762 18251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:17:18.831032 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 18251
I20260806 16:17:18.831216 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:17:18.831630 12246 external_mini_cluster.cc:1428] Setting key f018b8e71eeaf2996c1dc93e8c1b5d16
Aug 06 16:17:18 dist-test-slave-t3q3 krb5kdc[17684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033038, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:18.842507 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:18.827337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:33825 (local address 127.11.245.190:40697)
0806 16:17:18.827548 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:18.827554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:18.828816 (+ 1262us) server_negotiation.cc:407] Connection header received
0806 16:17:18.830454 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:18.830458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:18.830523 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:18.830634 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:18.832781 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:18.833543 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:18.835105 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:18.835369 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:18.835461 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:18.838438 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:18.838441 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:18.838444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:18.838472 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:18.839999 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:18.840659 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:18.840662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:18.840664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:18.840752 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:18.841189 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:18.841193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:18.841195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:18.841475 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:18.841573 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:18.842173 (+ 600us) server_negotiation.cc:299] Negotiation successful
0806 16:17:18.842343 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 16:17:18.843526 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:18.827919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:40697 (local address 127.11.245.131:33825)
0806 16:17:18.828597 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:18.828663 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:18.830183 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:18.831013 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:18.831024 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:18.831627 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:18.832529 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:18.832580 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:18.833683 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:18.833687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:18.834970 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:18.834983 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:18.835568 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:18.835572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:18.835742 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:18.836525 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:18.836544 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:18.838242 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:18.840178 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:18.840184 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:18.840200 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:18.840527 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:18.840871 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:18.840874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:18.840876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:18.841040 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:18.841593 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:18.841600 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:18.841894 (+ 294us) client_negotiation.cc:769] Sending connection context
0806 16:17:18.842078 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 16:17:18.842424 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":147,"threads_started":1}
I20260806 16:17:18.845036 18458 heartbeater.cc:347] Connected to a master server at 127.11.245.190:40697
I20260806 16:17:18.845396 18458 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:18.846094 18458 heartbeater.cc:511] Master 127.11.245.190:40697 requested a full tablet report, sending...
I20260806 16:17:18.847450 17746 ts_manager.cc:194] Registered new tserver with Master: a12bb3c6fdea4b13a2d4fa4eb914996f (127.11.245.131:43483)
I20260806 16:17:18.848207 17746 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:33825
I20260806 16:17:18.859297 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 16:17:18 dist-test-slave-t3q3 krb5kdc[17684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 16:17:18 dist-test-slave-t3q3 krb5kdc[17684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.129@KRBTEST.COM
I20260806 16:17:18.896273 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:18.882397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41636 (local address 127.11.245.129:41575)
0806 16:17:18.882766 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:18.882783 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:18.882821 (+ 38us) server_negotiation.cc:407] Connection header received
0806 16:17:18.882915 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:18.882921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:18.883173 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:18.883404 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:18.884523 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:18.885826 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:18.886903 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:18.887365 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:18.887472 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:18.890978 (+ 3506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:18.890996 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:18.891018 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:18.891059 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:18.893498 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:18.894171 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:18.894179 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:18.894181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:18.894304 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:18.894716 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:18.894720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:18.894721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:18.895167 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:18.895428 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:18.895734 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 16:17:18.896006 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":78,"threads_started":1}
Aug 06 16:17:18 dist-test-slave-t3q3 krb5kdc[17684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:18.912195 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:18.901087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46150 (local address 127.11.245.190:40697)
0806 16:17:18.901319 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:18.901326 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:18.901350 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:17:18.901546 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:18.901550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:18.901609 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:18.901728 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:18.902568 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:18.903463 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:18.904566 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:18.904802 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:18.904937 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:18.908211 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:18.908215 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:18.908218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:18.908260 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:18.910046 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:18.910570 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:18.910573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:18.910575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:18.910665 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:18.911044 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:18.911048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:18.911049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:18.911372 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:18.911512 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:18.911782 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 16:17:18.911910 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260806 16:17:18.912738 17746 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46150
I20260806 16:17:18.913604 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 17837
I20260806 16:17:18.942214 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 18044
I20260806 16:17:18.963706 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 18251
I20260806 16:17:18.980809 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 17703
2026-08-06T16:17:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2863 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:17:19 dist-test-slave-t3q3 krb5kdc[18475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:17:19 dist-test-slave-t3q3 krb5kdc[18475](info): set up 2 sockets
Aug 06 16:17:19 dist-test-slave-t3q3 krb5kdc[18475](info): commencing operation
krb5kdc: starting...
W20260806 16:17:20.283982 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:17:20 dist-test-slave-t3q3 krb5kdc[18475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:17:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:17:20.439514 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:33559
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:36933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:33559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:20.584951 18489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:20.585408 18489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:20.585492 18489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:20.590047 18489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:20.590180 18489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:20.590219 18489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:20.590260 18489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:20.590435 18489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:17:20.597488 18489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:36933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:33559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:33559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:20.599139 18489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:20.600569 18489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:20.610845 18500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:20.610852 18498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:20.610845 18502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:20.611505 18489 server_base.cc:1034] running on GCE node
I20260806 16:17:20.612444 18489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:20.614073 18489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:20.615294 18489 hybrid_clock.cc:648] HybridClock initialized: now 1786033040615273 us; error 42 us; skew 500 ppm
Aug 06 16:17:20 dist-test-slave-t3q3 krb5kdc[18475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033040, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:20.624368 18489 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:17:20.625519 18489 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:20.625638 18489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:20.625962 18489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:20.628947 18489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "f14b7a3097c34f8da9010ea6c82a23b2"
format_stamp: "Formatted at 2026-08-06 16:17:20 on dist-test-slave-t3q3"
server_key: "8e32e25884d11f8fedd2b40656171ee9"
server_key_iv: "4f2e5a6e0ce3d5e9009d535dbec86ad0"
server_key_version: "encryptionkey@0"
I20260806 16:17:20.629756 18489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "f14b7a3097c34f8da9010ea6c82a23b2"
format_stamp: "Formatted at 2026-08-06 16:17:20 on dist-test-slave-t3q3"
server_key: "8e32e25884d11f8fedd2b40656171ee9"
server_key_iv: "4f2e5a6e0ce3d5e9009d535dbec86ad0"
server_key_version: "encryptionkey@0"
I20260806 16:17:20.635121 18489 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:17:20.638327 18510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:20.640513 18489 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:17:20.640729 18489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "f14b7a3097c34f8da9010ea6c82a23b2"
format_stamp: "Formatted at 2026-08-06 16:17:20 on dist-test-slave-t3q3"
server_key: "8e32e25884d11f8fedd2b40656171ee9"
server_key_iv: "4f2e5a6e0ce3d5e9009d535dbec86ad0"
server_key_version: "encryptionkey@0"
I20260806 16:17:20.640935 18489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:20.655242 18489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:20.659205 18489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:20.659485 18489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:20.668839 18489 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:33559
I20260806 16:17:20.669191 18588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:33559 every 8 connection(s)
I20260806 16:17:20.669853 18489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:17:20.670728 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 18489
I20260806 16:17:20.670895 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:17:20.671264 12246 external_mini_cluster.cc:1428] Setting key a418c872aefb35a5c7f89e2c7c3d34c3
I20260806 16:17:20.674589 18589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 16:17:20 dist-test-slave-t3q3 krb5kdc[18475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:20.682578 18589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2: Bootstrap starting.
I20260806 16:17:20.685771 18589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:20.686751 18589 log.cc:824] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:20.688974 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:20.673319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46112 (local address 127.11.245.190:33559)
0806 16:17:20.674059 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:20.674080 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:20.674128 (+ 48us) server_negotiation.cc:407] Connection header received
0806 16:17:20.675074 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:20.675100 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:20.675542 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:20.675992 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:20.677037 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:20.678156 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:20.679118 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:20.679654 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:20.679960 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:20.683324 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:20.683330 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:20.683347 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:20.683382 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:20.685451 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:20.686003 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:20.686009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:20.686015 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:20.686138 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:20.686468 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:20.686473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:20.686475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:20.687043 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:20.687264 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:20.687624 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 16:17:20.687988 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":192,"threads_started":1}
I20260806 16:17:20.689381 18589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2: No bootstrap required, opened a new log
I20260806 16:17:20.694321 18589 raft_consensus.cc:348] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b7a3097c34f8da9010ea6c82a23b2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33559 } }
I20260806 16:17:20.694634 18589 raft_consensus.cc:374] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:20.694720 18589 raft_consensus.cc:729] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f14b7a3097c34f8da9010ea6c82a23b2, State: Initialized, Role: FOLLOWER
I20260806 16:17:20.695231 18589 consensus_queue.cc:260] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b7a3097c34f8da9010ea6c82a23b2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33559 } }
I20260806 16:17:20.695395 18589 raft_consensus.cc:388] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:17:20.695495 18589 raft_consensus.cc:482] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:17:20.695626 18589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:20.696828 18589 raft_consensus.cc:504] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b7a3097c34f8da9010ea6c82a23b2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33559 } }
I20260806 16:17:20.697273 18589 leader_election.cc:302] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f14b7a3097c34f8da9010ea6c82a23b2; no voters:
I20260806 16:17:20.697765 18589 leader_election.cc:288] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:17:20.697847 18596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:20.698096 18596 raft_consensus.cc:686] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [term 1 LEADER]: Becoming Leader. State: Replica: f14b7a3097c34f8da9010ea6c82a23b2, State: Running, Role: LEADER
I20260806 16:17:20.698467 18596 consensus_queue.cc:237] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b7a3097c34f8da9010ea6c82a23b2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33559 } }
I20260806 16:17:20.699419 18589 sys_catalog.cc:564] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:20.700628 18598 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f14b7a3097c34f8da9010ea6c82a23b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b7a3097c34f8da9010ea6c82a23b2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33559 } } }
I20260806 16:17:20.700779 18598 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:20.701172 18600 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f14b7a3097c34f8da9010ea6c82a23b2. Latest consensus state: current_term: 1 leader_uuid: "f14b7a3097c34f8da9010ea6c82a23b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b7a3097c34f8da9010ea6c82a23b2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33559 } } }
I20260806 16:17:20.701241 18611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:20.701363 18600 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2 [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:20.705479 18611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:20.713032 18611 catalog_manager.cc:1347] Generated new cluster ID: 159703014112446688ab27a7ae0ee39d
I20260806 16:17:20.713165 18611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:20.727610 18611 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:17:20.729413 18611 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:20.737505 18611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2: Generated new TSK 0
I20260806 16:17:20.738441 18611 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:17:20.784415 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33559
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:36933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:20.937480 18627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:20.937875 18627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:20.937963 18627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:20.942546 18627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:20.942628 18627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:20.942701 18627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:17:20.949105 18627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:36933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:20.950600 18627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:20.951936 18627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:20.963865 18640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:20.963857 18627 server_base.cc:1034] running on GCE node
W20260806 16:17:20.963840 18637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:20.963896 18636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:20.964799 18627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:20.965831 18627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:20.967083 18627 hybrid_clock.cc:648] HybridClock initialized: now 1786033040967031 us; error 73 us; skew 500 ppm
Aug 06 16:17:20 dist-test-slave-t3q3 krb5kdc[18475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033040, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:20.970715 18627 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:17:20.971691 18627 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:20.971763 18627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:20.971994 18627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:20.974754 18627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "08079a53b1eb4d238ee68e12e36bb1c8"
format_stamp: "Formatted at 2026-08-06 16:17:20 on dist-test-slave-t3q3"
server_key: "60c0d0a36155d38dc6da8fc78eb5f815"
server_key_iv: "b64482e3a836eb11f4755f64c34d77a6"
server_key_version: "encryptionkey@0"
I20260806 16:17:20.975441 18627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "08079a53b1eb4d238ee68e12e36bb1c8"
format_stamp: "Formatted at 2026-08-06 16:17:20 on dist-test-slave-t3q3"
server_key: "60c0d0a36155d38dc6da8fc78eb5f815"
server_key_iv: "b64482e3a836eb11f4755f64c34d77a6"
server_key_version: "encryptionkey@0"
I20260806 16:17:20.980341 18627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:17:20.983529 18648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:20.985219 18627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:17:20.985347 18627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "08079a53b1eb4d238ee68e12e36bb1c8"
format_stamp: "Formatted at 2026-08-06 16:17:20 on dist-test-slave-t3q3"
server_key: "60c0d0a36155d38dc6da8fc78eb5f815"
server_key_iv: "b64482e3a836eb11f4755f64c34d77a6"
server_key_version: "encryptionkey@0"
I20260806 16:17:20.985466 18627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:21.022490 18627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:21.026863 18627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:21.027091 18627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:21.027843 18627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:21.028980 18627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:21.029045 18627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:21.029098 18627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:21.029112 18627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:21.040987 18627 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:35765
I20260806 16:17:21.041011 18821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:35765 every 8 connection(s)
I20260806 16:17:21.041847 18627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:17:21.046670 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 18627
I20260806 16:17:21.046849 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:17:21.047281 12246 external_mini_cluster.cc:1428] Setting key 4aeafa894b7ff9a7ecf0a5eda49fd23f
Aug 06 16:17:21 dist-test-slave-t3q3 krb5kdc[18475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033040, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:17:21.062115 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.044435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:37945 (local address 127.11.245.190:33559)
0806 16:17:21.044671 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:21.044677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:21.046136 (+ 1459us) server_negotiation.cc:407] Connection header received
0806 16:17:21.048159 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:21.048163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:21.048239 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:21.048352 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:21.050129 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.050741 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.052652 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.052916 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.053050 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.056812 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:21.056816 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:21.056818 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:21.056853 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:21.058697 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:21.059577 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:21.059581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:21.059582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:21.059699 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:21.060310 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:21.060314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:21.060315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:21.060600 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:21.060694 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:21.061748 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0806 16:17:21.061942 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260806 16:17:21.063370 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.044999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:33559 (local address 127.11.245.129:37945)
0806 16:17:21.045868 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:21.045988 (+ 120us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:21.047857 (+ 1869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:21.048593 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:21.048602 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:21.049058 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:21.049910 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:21.049941 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.050898 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.050904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:21.052477 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:21.052491 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.053132 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.053136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:21.053338 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.054113 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:21.054134 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:21.056504 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:21.058867 (+ 2363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:21.058879 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:21.058910 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:21.059420 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:21.059816 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:21.059822 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:21.059825 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:21.060128 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:21.060752 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:21.060767 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:21.061331 (+ 564us) client_negotiation.cc:769] Sending connection context
0806 16:17:21.061664 (+ 333us) client_negotiation.cc:240] Negotiation successful
0806 16:17:21.062099 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":152,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
I20260806 16:17:21.065434 18822 heartbeater.cc:347] Connected to a master server at 127.11.245.190:33559
I20260806 16:17:21.065793 18822 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:21.066591 18822 heartbeater.cc:511] Master 127.11.245.190:33559 requested a full tablet report, sending...
I20260806 16:17:21.069170 18530 ts_manager.cc:194] Registered new tserver with Master: 08079a53b1eb4d238ee68e12e36bb1c8 (127.11.245.129:35765)
I20260806 16:17:21.071136 18530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:37945
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:17:21.085837 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33559
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:36933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:21.241222 18830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:21.241752 18830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:21.241819 18830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:21.246372 18830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:21.246451 18830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:21.246523 18830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:17:21.252784 18830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:36933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:21.254315 18830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:21.255620 18830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:21.267844 18836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:21.267844 18841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:21.267858 18837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:21.268180 18830 server_base.cc:1034] running on GCE node
I20260806 16:17:21.268843 18830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:21.269685 18830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:21.270901 18830 hybrid_clock.cc:648] HybridClock initialized: now 1786033041270861 us; error 50 us; skew 500 ppm
Aug 06 16:17:21 dist-test-slave-t3q3 krb5kdc[18475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033041, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:21.274633 18830 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:17:21.275708 18830 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:21.275822 18830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:21.276206 18830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:21.279134 18830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "66b0e345b9734c75bbd185ac961a869f"
format_stamp: "Formatted at 2026-08-06 16:17:21 on dist-test-slave-t3q3"
server_key: "3f641befc99550035dfd34e9af3d1336"
server_key_iv: "27e26dc12d522e7298d287aa1de12f77"
server_key_version: "encryptionkey@0"
I20260806 16:17:21.279935 18830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "66b0e345b9734c75bbd185ac961a869f"
format_stamp: "Formatted at 2026-08-06 16:17:21 on dist-test-slave-t3q3"
server_key: "3f641befc99550035dfd34e9af3d1336"
server_key_iv: "27e26dc12d522e7298d287aa1de12f77"
server_key_version: "encryptionkey@0"
I20260806 16:17:21.285015 18830 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:17:21.288554 18853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:21.290373 18830 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260806 16:17:21.290561 18830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "66b0e345b9734c75bbd185ac961a869f"
format_stamp: "Formatted at 2026-08-06 16:17:21 on dist-test-slave-t3q3"
server_key: "3f641befc99550035dfd34e9af3d1336"
server_key_iv: "27e26dc12d522e7298d287aa1de12f77"
server_key_version: "encryptionkey@0"
I20260806 16:17:21.290712 18830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:21.308037 18830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:21.312145 18830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:21.312448 18830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:21.313511 18830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:21.314607 18830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:21.314659 18830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:21.314707 18830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:21.314723 18830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:21.326543 18830 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:34457
I20260806 16:17:21.326561 19025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:34457 every 8 connection(s)
I20260806 16:17:21.327374 18830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:17:21.337352 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 18830
I20260806 16:17:21.337539 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:17:21.337958 12246 external_mini_cluster.cc:1428] Setting key 154e31c5e3bf7a2977d71ec38517391c
Aug 06 16:17:21 dist-test-slave-t3q3 krb5kdc[18475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033041, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:21.345243 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.330021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:35551 (local address 127.11.245.190:33559)
0806 16:17:21.330223 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:21.330229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:21.331340 (+ 1111us) server_negotiation.cc:407] Connection header received
0806 16:17:21.332948 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:21.332951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:21.333014 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:21.333110 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:21.334596 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.335174 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.336530 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.336786 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.336902 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.340635 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:21.340639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:21.340641 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:21.340667 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:21.342527 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:21.343167 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:21.343171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:21.343173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:21.343276 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:21.343697 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:21.343700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:21.343702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:21.343955 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:21.344114 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:21.344846 (+ 732us) server_negotiation.cc:299] Negotiation successful
0806 16:17:21.345027 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 16:17:21.346402 19029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.330536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:33559 (local address 127.11.245.130:35551)
0806 16:17:21.331144 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:21.331198 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:21.332703 (+ 1505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:21.333334 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:21.333342 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:21.333800 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:21.334446 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:21.334463 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.335299 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.335302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:21.336389 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:21.336401 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.337069 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.337074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:21.337345 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.338259 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:21.338278 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:21.340432 (+ 2154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:21.342676 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:21.342684 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:21.342709 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:21.343033 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:21.343400 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:21.343403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:21.343408 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:21.343582 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:21.344128 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:21.344139 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:21.344498 (+ 359us) client_negotiation.cc:769] Sending connection context
0806 16:17:21.344734 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 16:17:21.345190 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":140,"threads_started":1}
I20260806 16:17:21.348006 19026 heartbeater.cc:347] Connected to a master server at 127.11.245.190:33559
I20260806 16:17:21.348433 19026 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:21.349287 19026 heartbeater.cc:511] Master 127.11.245.190:33559 requested a full tablet report, sending...
I20260806 16:17:21.350983 18530 ts_manager.cc:194] Registered new tserver with Master: 66b0e345b9734c75bbd185ac961a869f (127.11.245.130:34457)
I20260806 16:17:21.352111 18530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:35551
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:17:21.376219 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33559
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:36933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:21.521906 19033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:21.522279 19033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:21.522333 19033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:21.526871 19033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:21.526976 19033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:21.527052 19033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:17:21.533775 19033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:36933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:21.535600 19033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:21.537074 19033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:17:21.549726 19033 server_base.cc:1034] running on GCE node
W20260806 16:17:21.549664 19041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:21.549649 19043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:21.549639 19040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:21.550433 19033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:21.551263 19033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:21.552451 19033 hybrid_clock.cc:648] HybridClock initialized: now 1786033041552418 us; error 43 us; skew 500 ppm
Aug 06 16:17:21 dist-test-slave-t3q3 krb5kdc[18475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033041, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:21.556205 19033 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:17:21.557327 19033 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:21.557446 19033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:21.557765 19033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:21.560664 19033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "d4e832dfdb8d4415b81545f45455369b"
format_stamp: "Formatted at 2026-08-06 16:17:21 on dist-test-slave-t3q3"
server_key: "ea31fd1354c1dfe7214b36434fe22ccb"
server_key_iv: "e36f4190e931eb4a930fc56bff05f4bb"
server_key_version: "encryptionkey@0"
I20260806 16:17:21.561416 19033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "d4e832dfdb8d4415b81545f45455369b"
format_stamp: "Formatted at 2026-08-06 16:17:21 on dist-test-slave-t3q3"
server_key: "ea31fd1354c1dfe7214b36434fe22ccb"
server_key_iv: "e36f4190e931eb4a930fc56bff05f4bb"
server_key_version: "encryptionkey@0"
I20260806 16:17:21.566443 19033 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:17:21.569955 19052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:21.571698 19033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:21.571897 19033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "d4e832dfdb8d4415b81545f45455369b"
format_stamp: "Formatted at 2026-08-06 16:17:21 on dist-test-slave-t3q3"
server_key: "ea31fd1354c1dfe7214b36434fe22ccb"
server_key_iv: "e36f4190e931eb4a930fc56bff05f4bb"
server_key_version: "encryptionkey@0"
I20260806 16:17:21.572067 19033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:21.585268 19033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:21.589618 19033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:21.589870 19033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:21.590708 19033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:21.591810 19033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:21.591883 19033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:21.591966 19033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:21.592020 19033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:21.603416 19033 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:45773
I20260806 16:17:21.603500 19229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:45773 every 8 connection(s)
I20260806 16:17:21.604317 19033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:17:21.607019 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 19033
I20260806 16:17:21.607195 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:17:21.607610 12246 external_mini_cluster.cc:1428] Setting key c01bd7397eebf5cd0b611c6965c806e1
Aug 06 16:17:21 dist-test-slave-t3q3 krb5kdc[18475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033041, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:21.622663 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.606810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:49799 (local address 127.11.245.190:33559)
0806 16:17:21.607012 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:21.607017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:21.608376 (+ 1359us) server_negotiation.cc:407] Connection header received
0806 16:17:21.610437 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:21.610443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:21.610513 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:21.610579 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:21.612179 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.612813 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.614245 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.614487 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.614594 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.617963 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:21.617967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:21.617970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:21.618004 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:21.619856 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:21.620580 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:21.620584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:21.620585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:21.620670 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:21.621081 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:21.621084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:21.621086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:21.621409 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:21.621532 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:21.622316 (+ 784us) server_negotiation.cc:299] Negotiation successful
0806 16:17:21.622510 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 16:17:21.624051 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.607378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:33559 (local address 127.11.245.131:49799)
0806 16:17:21.608125 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:21.608203 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:21.610135 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:21.610830 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:21.610839 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:21.611312 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:21.611972 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:21.611989 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.612972 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.612976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:21.614096 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:21.614108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.614717 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.614724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:21.614930 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.615731 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:21.615752 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:21.617731 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:21.620016 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:21.620024 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:21.620054 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:21.620450 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:21.620790 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:21.620793 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:21.620795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:21.620968 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:21.621530 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:21.621540 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:21.621976 (+ 436us) client_negotiation.cc:769] Sending connection context
0806 16:17:21.622222 (+ 246us) client_negotiation.cc:240] Negotiation successful
0806 16:17:21.622686 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":173,"threads_started":1}
I20260806 16:17:21.625643 19234 heartbeater.cc:347] Connected to a master server at 127.11.245.190:33559
I20260806 16:17:21.625955 19234 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:21.626570 19234 heartbeater.cc:511] Master 127.11.245.190:33559 requested a full tablet report, sending...
I20260806 16:17:21.627938 18530 ts_manager.cc:194] Registered new tserver with Master: d4e832dfdb8d4415b81545f45455369b (127.11.245.131:45773)
I20260806 16:17:21.628698 18530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:49799
I20260806 16:17:21.635286 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:17:21.648034 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.637895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46120 (local address 127.11.245.190:33559)
0806 16:17:21.638099 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:21.638106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:21.638282 (+ 176us) server_negotiation.cc:407] Connection header received
0806 16:17:21.638487 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:21.638491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:21.638549 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:21.638640 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:21.639487 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.640121 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.641011 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.641249 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.641362 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.643308 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:21.643313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:21.643316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:21.643348 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:21.645592 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:21.646326 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:21.646331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:21.646333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:21.646485 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:21.646847 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:21.646852 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:21.646854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:21.647172 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:21.647356 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:21.647656 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 16:17:21.647810 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 16:17:21.651818 18530 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:17:21.655742 18530 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:17:21.675017 19256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.667236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:45773 (local address 127.0.0.1:48916)
0806 16:17:21.668446 (+ 1210us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:21.668510 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:21.668678 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:21.669650 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:21.669660 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:21.669689 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:21.669859 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:21.669868 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.671493 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.671497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:21.672983 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:21.672994 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.674409 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.674413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:21.674528 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.674567 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 16:17:21.674682 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 16:17:21.674837 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":962,"mutex_wait_us":1,"thread_start_us":149,"threads_started":1}
I20260806 16:17:21.675307 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.668395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37376 (local address 127.11.245.129:35765)
0806 16:17:21.668722 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:21.668732 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:21.668999 (+ 267us) server_negotiation.cc:407] Connection header received
0806 16:17:21.669271 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:21.669276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:21.669474 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:21.669618 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:21.670148 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.671331 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.673483 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.674348 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.674458 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.674597 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:21.674999 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 16:17:21.675136 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":99,"threads_started":1}
I20260806 16:17:21.675699 19255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.667310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:35765 (local address 127.0.0.1:37376)
0806 16:17:21.668885 (+ 1575us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:21.668900 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:21.669086 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:21.669845 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:21.669849 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:21.669869 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:21.670004 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:21.670010 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.671483 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.671487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:21.673334 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:21.673345 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.674707 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.674710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:21.674824 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.674847 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 16:17:21.675493 (+ 646us) client_negotiation.cc:240] Negotiation successful
0806 16:17:21.675576 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1507,"mutex_wait_us":590,"thread_start_us":113,"threads_started":1}
I20260806 16:17:21.675303 19260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.668865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:34457 (local address 127.0.0.1:44212)
0806 16:17:21.669898 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:21.669911 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:21.669998 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:21.671005 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:21.671009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:21.671026 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:21.671119 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:21.671124 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.672517 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.672520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:21.673635 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:21.673645 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.674908 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.674912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:21.675004 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.675026 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 16:17:21.675087 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 16:17:21.675156 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"thread_start_us":114,"threads_started":1}
I20260806 16:17:21.676318 19257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.667585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48916 (local address 127.11.245.131:45773)
0806 16:17:21.668008 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:21.668024 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:21.668877 (+ 853us) server_negotiation.cc:407] Connection header received
0806 16:17:21.668989 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:21.668995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:21.669245 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:21.669415 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:21.670016 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.671336 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.673151 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.674060 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.674189 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.674345 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:21.676000 (+ 1655us) server_negotiation.cc:299] Negotiation successful
0806 16:17:21.676149 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":165,"threads_started":1}
I20260806 16:17:21.675773 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.669426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44212 (local address 127.11.245.130:34457)
0806 16:17:21.670300 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:21.670313 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:21.670338 (+ 25us) server_negotiation.cc:407] Connection header received
0806 16:17:21.670468 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:21.670473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:21.670662 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:21.670812 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:21.671239 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.672358 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.673768 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.674590 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.674713 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.674841 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:21.675499 (+ 658us) server_negotiation.cc:299] Negotiation successful
0806 16:17:21.675609 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":730,"thread_start_us":161,"threads_started":1}
I20260806 16:17:21.679736 18924 tablet_service.cc:1467] Processing CreateTablet for tablet cb296b562160428a84f42f170df21a7d (DEFAULT_TABLE table=test-table [id=11de4519597741aa9409ec5e9b0e0cfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:21.679734 18722 tablet_service.cc:1467] Processing CreateTablet for tablet cb296b562160428a84f42f170df21a7d (DEFAULT_TABLE table=test-table [id=11de4519597741aa9409ec5e9b0e0cfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:21.679738 19131 tablet_service.cc:1467] Processing CreateTablet for tablet cb296b562160428a84f42f170df21a7d (DEFAULT_TABLE table=test-table [id=11de4519597741aa9409ec5e9b0e0cfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:21.681241 18924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb296b562160428a84f42f170df21a7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:21.681267 18722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb296b562160428a84f42f170df21a7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:21.681893 19131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb296b562160428a84f42f170df21a7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:21.690608 19264 tablet_bootstrap.cc:492] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8: Bootstrap starting.
I20260806 16:17:21.691535 19266 tablet_bootstrap.cc:492] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f: Bootstrap starting.
I20260806 16:17:21.692235 19267 tablet_bootstrap.cc:492] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b: Bootstrap starting.
I20260806 16:17:21.694186 19266 tablet_bootstrap.cc:654] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:21.694188 19264 tablet_bootstrap.cc:654] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:21.694674 19267 tablet_bootstrap.cc:654] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:21.695456 19264 log.cc:824] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:21.695798 19266 log.cc:824] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:21.696009 19267 log.cc:824] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:21.698107 19264 tablet_bootstrap.cc:492] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8: No bootstrap required, opened a new log
I20260806 16:17:21.698107 19266 tablet_bootstrap.cc:492] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f: No bootstrap required, opened a new log
I20260806 16:17:21.698107 19267 tablet_bootstrap.cc:492] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b: No bootstrap required, opened a new log
I20260806 16:17:21.698369 19264 ts_tablet_manager.cc:1397] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260806 16:17:21.698369 19267 ts_tablet_manager.cc:1397] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 16:17:21.698369 19266 ts_tablet_manager.cc:1397] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 16:17:21.703290 19264 raft_consensus.cc:348] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e832dfdb8d4415b81545f45455369b" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 45773 } } peers { permanent_uuid: "08079a53b1eb4d238ee68e12e36bb1c8" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 35765 } } peers { permanent_uuid: "66b0e345b9734c75bbd185ac961a869f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 34457 } }
I20260806 16:17:21.703290 19267 raft_consensus.cc:348] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e832dfdb8d4415b81545f45455369b" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 45773 } } peers { permanent_uuid: "08079a53b1eb4d238ee68e12e36bb1c8" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 35765 } } peers { permanent_uuid: "66b0e345b9734c75bbd185ac961a869f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 34457 } }
I20260806 16:17:21.703290 19266 raft_consensus.cc:348] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e832dfdb8d4415b81545f45455369b" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 45773 } } peers { permanent_uuid: "08079a53b1eb4d238ee68e12e36bb1c8" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 35765 } } peers { permanent_uuid: "66b0e345b9734c75bbd185ac961a869f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 34457 } }
I20260806 16:17:21.703665 19267 raft_consensus.cc:374] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:21.703663 19266 raft_consensus.cc:374] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:21.703665 19264 raft_consensus.cc:374] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:21.703733 19264 raft_consensus.cc:729] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08079a53b1eb4d238ee68e12e36bb1c8, State: Initialized, Role: FOLLOWER
I20260806 16:17:21.703733 19266 raft_consensus.cc:729] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b0e345b9734c75bbd185ac961a869f, State: Initialized, Role: FOLLOWER
I20260806 16:17:21.703733 19267 raft_consensus.cc:729] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e832dfdb8d4415b81545f45455369b, State: Initialized, Role: FOLLOWER
I20260806 16:17:21.704388 19264 consensus_queue.cc:260] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e832dfdb8d4415b81545f45455369b" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 45773 } } peers { permanent_uuid: "08079a53b1eb4d238ee68e12e36bb1c8" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 35765 } } peers { permanent_uuid: "66b0e345b9734c75bbd185ac961a869f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 34457 } }
I20260806 16:17:21.704388 19267 consensus_queue.cc:260] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e832dfdb8d4415b81545f45455369b" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 45773 } } peers { permanent_uuid: "08079a53b1eb4d238ee68e12e36bb1c8" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 35765 } } peers { permanent_uuid: "66b0e345b9734c75bbd185ac961a869f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 34457 } }
I20260806 16:17:21.704388 19266 consensus_queue.cc:260] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e832dfdb8d4415b81545f45455369b" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 45773 } } peers { permanent_uuid: "08079a53b1eb4d238ee68e12e36bb1c8" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 35765 } } peers { permanent_uuid: "66b0e345b9734c75bbd185ac961a869f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 34457 } }
I20260806 16:17:21.705816 19267 ts_tablet_manager.cc:1428] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 16:17:21.705822 19266 ts_tablet_manager.cc:1428] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 16:17:21.705818 19264 ts_tablet_manager.cc:1428] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 16:17:21.705994 19026 heartbeater.cc:503] Master 127.11.245.190:33559 was elected leader, sending a full tablet report...
I20260806 16:17:21.706327 19234 heartbeater.cc:503] Master 127.11.245.190:33559 was elected leader, sending a full tablet report...
I20260806 16:17:21.705818 18822 heartbeater.cc:503] Master 127.11.245.190:33559 was elected leader, sending a full tablet report...
W20260806 16:17:21.793602 18823 tablet.cc:2369] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:17:21.829169 19027 tablet.cc:2369] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:17:21.855574 19235 tablet.cc:2369] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:17:21.875245 19280 raft_consensus.cc:482] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:21.875578 19280 raft_consensus.cc:504] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e832dfdb8d4415b81545f45455369b" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 45773 } } peers { permanent_uuid: "08079a53b1eb4d238ee68e12e36bb1c8" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 35765 } } peers { permanent_uuid: "66b0e345b9734c75bbd185ac961a869f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 34457 } }
I20260806 16:17:21.877493 19280 leader_election.cc:288] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4e832dfdb8d4415b81545f45455369b (127.11.245.131:45773), 66b0e345b9734c75bbd185ac961a869f (127.11.245.130:34457)
I20260806 16:17:21.882726 19257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.877926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:39727 (local address 127.11.245.131:45773)
0806 16:17:21.878116 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:21.878122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:21.878432 (+ 310us) server_negotiation.cc:407] Connection header received
0806 16:17:21.878647 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:21.878650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:21.878703 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:21.878791 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:21.879182 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.879749 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.881431 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.882026 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.882172 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.882244 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:21.882456 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 16:17:21.882550 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 16:17:21.882726 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.878031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:46699 (local address 127.11.245.130:34457)
0806 16:17:21.878331 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:21.878337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:21.878547 (+ 210us) server_negotiation.cc:407] Connection header received
0806 16:17:21.878722 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:21.878727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:21.878782 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:21.878873 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:21.879192 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.879749 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.881445 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.882001 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.882150 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.882225 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:21.882456 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 16:17:21.882548 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260806 16:17:21.882762 19281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.877885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:34457 (local address 127.11.245.129:46699)
0806 16:17:21.878427 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:21.878444 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:21.878596 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:21.878888 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:21.878891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:21.878938 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:21.879073 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:21.879078 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.880010 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.880013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:21.881257 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:21.881270 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.882147 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.882151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:21.882288 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.882328 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 16:17:21.882432 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 16:17:21.882548 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":204,"thread_start_us":118,"threads_started":1}
I20260806 16:17:21.882766 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.877811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:45773 (local address 127.11.245.129:39727)
0806 16:17:21.878306 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:21.878321 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:21.878495 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:21.878818 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:21.878821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:21.878839 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:21.879055 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:21.879062 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.880013 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.880020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:21.881258 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:21.881270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.882160 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.882163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:21.882325 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.882344 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 16:17:21.882437 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 16:17:21.882548 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":139,"spinlock_wait_cycles":91904,"thread_start_us":186,"threads_started":1}
I20260806 16:17:21.883816 18953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb296b562160428a84f42f170df21a7d" candidate_uuid: "08079a53b1eb4d238ee68e12e36bb1c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b0e345b9734c75bbd185ac961a869f" is_pre_election: true
I20260806 16:17:21.884106 19155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb296b562160428a84f42f170df21a7d" candidate_uuid: "08079a53b1eb4d238ee68e12e36bb1c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e832dfdb8d4415b81545f45455369b" is_pre_election: true
I20260806 16:17:21.884341 18953 raft_consensus.cc:2454] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08079a53b1eb4d238ee68e12e36bb1c8 in term 0.
I20260806 16:17:21.884466 19155 raft_consensus.cc:2454] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08079a53b1eb4d238ee68e12e36bb1c8 in term 0.
I20260806 16:17:21.885043 18653 leader_election.cc:302] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08079a53b1eb4d238ee68e12e36bb1c8, d4e832dfdb8d4415b81545f45455369b; no voters:
I20260806 16:17:21.885373 19280 raft_consensus.cc:2781] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:21.885485 19280 raft_consensus.cc:482] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:21.885535 19280 raft_consensus.cc:3037] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:21.886965 19280 raft_consensus.cc:504] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e832dfdb8d4415b81545f45455369b" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 45773 } } peers { permanent_uuid: "08079a53b1eb4d238ee68e12e36bb1c8" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 35765 } } peers { permanent_uuid: "66b0e345b9734c75bbd185ac961a869f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 34457 } }
I20260806 16:17:21.887423 19280 leader_election.cc:288] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [CANDIDATE]: Term 1 election: Requested vote from peers d4e832dfdb8d4415b81545f45455369b (127.11.245.131:45773), 66b0e345b9734c75bbd185ac961a869f (127.11.245.130:34457)
I20260806 16:17:21.887857 19155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb296b562160428a84f42f170df21a7d" candidate_uuid: "08079a53b1eb4d238ee68e12e36bb1c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4e832dfdb8d4415b81545f45455369b"
I20260806 16:17:21.888056 19155 raft_consensus.cc:3037] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:21.888206 18953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb296b562160428a84f42f170df21a7d" candidate_uuid: "08079a53b1eb4d238ee68e12e36bb1c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b0e345b9734c75bbd185ac961a869f"
I20260806 16:17:21.888401 18953 raft_consensus.cc:3037] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:21.889600 19155 raft_consensus.cc:2454] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08079a53b1eb4d238ee68e12e36bb1c8 in term 1.
I20260806 16:17:21.889979 18653 leader_election.cc:302] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08079a53b1eb4d238ee68e12e36bb1c8, d4e832dfdb8d4415b81545f45455369b; no voters:
I20260806 16:17:21.890105 18953 raft_consensus.cc:2454] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08079a53b1eb4d238ee68e12e36bb1c8 in term 1.
I20260806 16:17:21.890269 19280 raft_consensus.cc:2781] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:21.890691 19280 raft_consensus.cc:686] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [term 1 LEADER]: Becoming Leader. State: Replica: 08079a53b1eb4d238ee68e12e36bb1c8, State: Running, Role: LEADER
I20260806 16:17:21.891238 19280 consensus_queue.cc:237] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e832dfdb8d4415b81545f45455369b" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 45773 } } peers { permanent_uuid: "08079a53b1eb4d238ee68e12e36bb1c8" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 35765 } } peers { permanent_uuid: "66b0e345b9734c75bbd185ac961a869f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 34457 } }
I20260806 16:17:21.896153 18530 catalog_manager.cc:5495] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08079a53b1eb4d238ee68e12e36bb1c8 (127.11.245.129). New cstate: current_term: 1 leader_uuid: "08079a53b1eb4d238ee68e12e36bb1c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4e832dfdb8d4415b81545f45455369b" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 45773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08079a53b1eb4d238ee68e12e36bb1c8" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 35765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66b0e345b9734c75bbd185ac961a869f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 34457 } health_report { overall_health: UNKNOWN } } }
I20260806 16:17:21.927495 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.922935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37380 (local address 127.11.245.129:35765)
0806 16:17:21.923106 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:21.923111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:21.923126 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:17:21.923239 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:21.923242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:21.923324 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:21.923433 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:21.923768 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.924433 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.925438 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:21.925692 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:21.925791 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:21.926227 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:21.926929 (+ 702us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:21.927050 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:21.927230 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 16:17:21.927312 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 16:17:21.936134 18953 raft_consensus.cc:1261] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f [term 1 FOLLOWER]: Refusing update from remote peer 08079a53b1eb4d238ee68e12e36bb1c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:21.936134 19155 raft_consensus.cc:1261] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b [term 1 FOLLOWER]: Refusing update from remote peer 08079a53b1eb4d238ee68e12e36bb1c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:21.937170 19283 consensus_queue.cc:1035] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b0e345b9734c75bbd185ac961a869f" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 34457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:21.937480 19280 consensus_queue.cc:1035] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4e832dfdb8d4415b81545f45455369b" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 45773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:21.947001 19290 mvcc.cc:205] Tried to move back new op lower bound from 7315591339759575040 to 7315591339590942720. Current Snapshot: MvccSnapshot[applied={T|T < 7315591339759575040}]
I20260806 16:17:21.948483 19288 mvcc.cc:205] Tried to move back new op lower bound from 7315591339759575040 to 7315591339590942720. Current Snapshot: MvccSnapshot[applied={T|T < 7315591339759575040}]
I20260806 16:17:21.957788 18537 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:21.958127 18537 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:21.961899 18537 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f14b7a3097c34f8da9010ea6c82a23b2: Sending DeleteTablet for 3 replicas of tablet cb296b562160428a84f42f170df21a7d
I20260806 16:17:21.963460 18924 tablet_service.cc:1514] Processing DeleteTablet for tablet cb296b562160428a84f42f170df21a7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:21 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:44212
I20260806 16:17:21.963464 19131 tablet_service.cc:1514] Processing DeleteTablet for tablet cb296b562160428a84f42f170df21a7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:21 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:48916
I20260806 16:17:21.963783 18722 tablet_service.cc:1514] Processing DeleteTablet for tablet cb296b562160428a84f42f170df21a7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:21 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:37376
I20260806 16:17:21.964280 19298 tablet_replica.cc:330] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f: stopping tablet replica
I20260806 16:17:21.964545 19302 tablet_replica.cc:330] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8: stopping tablet replica
I20260806 16:17:21.964782 19298 raft_consensus.cc:2229] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:21.964876 19302 raft_consensus.cc:2229] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:17:21.965281 19298 raft_consensus.cc:2258] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:21.965307 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 18627
I20260806 16:17:21.965409 19302 raft_consensus.cc:2258] T cb296b562160428a84f42f170df21a7d P 08079a53b1eb4d238ee68e12e36bb1c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:21.965741 19301 tablet_replica.cc:330] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b: stopping tablet replica
I20260806 16:17:21.966158 19301 raft_consensus.cc:2229] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:21.966481 19301 raft_consensus.cc:2258] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:21.971441 19298 ts_tablet_manager.cc:1905] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:21.973078 19301 ts_tablet_manager.cc:1905] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:21.975173 19298 ts_tablet_manager.cc:1918] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:21.975322 19298 log.cc:1196] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/cb296b562160428a84f42f170df21a7d
I20260806 16:17:21.975713 19298 ts_tablet_manager.cc:1939] T cb296b562160428a84f42f170df21a7d P 66b0e345b9734c75bbd185ac961a869f: Deleting consensus metadata
I20260806 16:17:21.975950 19301 ts_tablet_manager.cc:1918] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:21.976104 19301 log.cc:1196] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/cb296b562160428a84f42f170df21a7d
I20260806 16:17:21.976497 19301 ts_tablet_manager.cc:1939] T cb296b562160428a84f42f170df21a7d P d4e832dfdb8d4415b81545f45455369b: Deleting consensus metadata
I20260806 16:17:21.977837 18515 catalog_manager.cc:4841] TS 66b0e345b9734c75bbd185ac961a869f (127.11.245.130:34457): tablet cb296b562160428a84f42f170df21a7d (table test-table [id=11de4519597741aa9409ec5e9b0e0cfe]) successfully deleted
I20260806 16:17:21.977839 18516 catalog_manager.cc:4841] TS d4e832dfdb8d4415b81545f45455369b (127.11.245.131:45773): tablet cb296b562160428a84f42f170df21a7d (table test-table [id=11de4519597741aa9409ec5e9b0e0cfe]) successfully deleted
W20260806 16:17:21.982316 18515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:35765 (error 108)
I20260806 16:17:21.982575 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 18830
I20260806 16:17:21.995702 19256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:21.995074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:35765 (local address 127.0.0.1:37394)
0806 16:17:21.995427 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:21.995544 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:35765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219}
W20260806 16:17:21.996171 18515 catalog_manager.cc:4584] TS 08079a53b1eb4d238ee68e12e36bb1c8 (127.11.245.129:35765): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cb296b562160428a84f42f170df21a7d: Network error: Client connection negotiation failed: client connection to 127.11.245.129:35765: connect: Connection refused (error 111)
I20260806 16:17:22.005813 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 19033
I20260806 16:17:22.028674 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 18489
2026-08-06T16:17:22Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3045 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T16:17:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:22Z Disabled control of system clock
I20260806 16:17:22.074347 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:43023
--webserver_interface=127.11.245.190
--webserver_port=0
--builtin_ntp_servers=127.11.245.148:33151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:43023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 16:17:22.219808 19310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:22.220266 19310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:22.220314 19310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:22.225229 19310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:22.225363 19310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:22.225382 19310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:22.225404 19310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:22.225421 19310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:17:22.231712 19310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:43023
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:43023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:22.233376 19310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:22.235173 19310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:22.245497 19318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:22.245613 19327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:22.245539 19323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:22.245739 19310 server_base.cc:1034] running on GCE node
I20260806 16:17:22.246553 19310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:22.248021 19310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:22.249303 19310 hybrid_clock.cc:648] HybridClock initialized: now 1786033042249267 us; error 48 us; skew 500 ppm
I20260806 16:17:22.251914 19310 webserver.cc:460] Webserver started at http://127.11.245.190:33383/ using document root <none> and password file <none>
I20260806 16:17:22.252738 19310 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:22.252864 19310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:22.253193 19310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:22.255980 19310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "fd2eb47c950f4c01af0402a78a2f767d"
format_stamp: "Formatted at 2026-08-06 16:17:22 on dist-test-slave-t3q3"
server_key: "07222248cd6c7415f7d0c673a3563929"
server_key_iv: "11880c282f15f1ea299d34693fe83e40"
server_key_version: "encryptionkey@0"
I20260806 16:17:22.256861 19310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "fd2eb47c950f4c01af0402a78a2f767d"
format_stamp: "Formatted at 2026-08-06 16:17:22 on dist-test-slave-t3q3"
server_key: "07222248cd6c7415f7d0c673a3563929"
server_key_iv: "11880c282f15f1ea299d34693fe83e40"
server_key_version: "encryptionkey@0"
I20260806 16:17:22.262092 19310 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:17:22.266075 19336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:22.267846 19310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 16:17:22.268103 19310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "fd2eb47c950f4c01af0402a78a2f767d"
format_stamp: "Formatted at 2026-08-06 16:17:22 on dist-test-slave-t3q3"
server_key: "07222248cd6c7415f7d0c673a3563929"
server_key_iv: "11880c282f15f1ea299d34693fe83e40"
server_key_version: "encryptionkey@0"
I20260806 16:17:22.268294 19310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:22.274024 19310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:22.274806 19310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:22.275023 19310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:22.284300 19310 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:43023
I20260806 16:17:22.284577 19412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:43023 every 8 connection(s)
I20260806 16:17:22.285655 19310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:17:22.290467 19414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:22.293260 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 19310
I20260806 16:17:22.293414 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:17:22.293807 12246 external_mini_cluster.cc:1428] Setting key 2d080862e7465e3fddfaec59897c1303
I20260806 16:17:22.298728 19414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d: Bootstrap starting.
I20260806 16:17:22.300974 19419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:22.295852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44020 (local address 127.11.245.190:43023)
0806 16:17:22.296699 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:22.296718 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:22.296766 (+ 48us) server_negotiation.cc:407] Connection header received
0806 16:17:22.297528 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:22.297551 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:22.298006 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:22.298423 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:22.299349 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:22.299354 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:22.299368 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:17:22.299378 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:22.299444 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:17:22.299462 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:22.299652 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:22.299741 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 16:17:22.300044 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":188,"threads_started":1}
I20260806 16:17:22.301970 19414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:22.302913 19414 log.cc:824] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:22.306674 19414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d: No bootstrap required, opened a new log
I20260806 16:17:22.311605 19414 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2eb47c950f4c01af0402a78a2f767d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43023 } }
I20260806 16:17:22.311978 19414 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:22.312103 19414 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd2eb47c950f4c01af0402a78a2f767d, State: Initialized, Role: FOLLOWER
I20260806 16:17:22.312857 19414 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2eb47c950f4c01af0402a78a2f767d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43023 } }
I20260806 16:17:22.313084 19414 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:17:22.313186 19414 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:17:22.313328 19414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:22.314589 19414 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2eb47c950f4c01af0402a78a2f767d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43023 } }
I20260806 16:17:22.315048 19414 leader_election.cc:302] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd2eb47c950f4c01af0402a78a2f767d; no voters:
I20260806 16:17:22.315505 19414 leader_election.cc:288] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:17:22.315724 19423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:22.316042 19423 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [term 1 LEADER]: Becoming Leader. State: Replica: fd2eb47c950f4c01af0402a78a2f767d, State: Running, Role: LEADER
I20260806 16:17:22.316498 19423 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2eb47c950f4c01af0402a78a2f767d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43023 } }
I20260806 16:17:22.317229 19414 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:22.318797 19425 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd2eb47c950f4c01af0402a78a2f767d. Latest consensus state: current_term: 1 leader_uuid: "fd2eb47c950f4c01af0402a78a2f767d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2eb47c950f4c01af0402a78a2f767d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43023 } } }
I20260806 16:17:22.318950 19425 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:22.319093 19424 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd2eb47c950f4c01af0402a78a2f767d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2eb47c950f4c01af0402a78a2f767d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43023 } } }
I20260806 16:17:22.319242 19424 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:22.319463 19433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:22.322988 19433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:22.329813 19433 catalog_manager.cc:1347] Generated new cluster ID: 6a6a0ceea4fd472684868a4f7aedb5b4
I20260806 16:17:22.329907 19433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:22.345274 19433 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:17:22.345811 19433 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:22.353564 19433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d: Generated new TSK 0
I20260806 16:17:22.354575 19433 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:17:22.370064 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43023
--builtin_ntp_servers=127.11.245.148:33151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 16:17:22.517776 19445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:22.518184 19445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:22.518267 19445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:22.522984 19445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:22.523082 19445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:22.523151 19445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:17:22.529402 19445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:22.531159 19445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:22.533151 19445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:22.545621 19457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:22.545622 19453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:22.545643 19452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:22.546066 19445 server_base.cc:1034] running on GCE node
I20260806 16:17:22.546712 19445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:22.547571 19445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:22.548770 19445 hybrid_clock.cc:648] HybridClock initialized: now 1786033042548712 us; error 74 us; skew 500 ppm
I20260806 16:17:22.551262 19445 webserver.cc:460] Webserver started at http://127.11.245.129:34967/ using document root <none> and password file <none>
I20260806 16:17:22.552111 19445 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:22.552214 19445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:22.552508 19445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:22.555302 19445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "8a99dd9e4eb944d6927aea30b93eca31"
format_stamp: "Formatted at 2026-08-06 16:17:22 on dist-test-slave-t3q3"
server_key: "4f635ed5e95f6a903559804ae952794a"
server_key_iv: "1011556b48f440c7e79a15d0fce9f176"
server_key_version: "encryptionkey@0"
I20260806 16:17:22.556035 19445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "8a99dd9e4eb944d6927aea30b93eca31"
format_stamp: "Formatted at 2026-08-06 16:17:22 on dist-test-slave-t3q3"
server_key: "4f635ed5e95f6a903559804ae952794a"
server_key_iv: "1011556b48f440c7e79a15d0fce9f176"
server_key_version: "encryptionkey@0"
I20260806 16:17:22.561064 19445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:17:22.564975 19464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:22.566893 19445 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:17:22.567281 19445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "8a99dd9e4eb944d6927aea30b93eca31"
format_stamp: "Formatted at 2026-08-06 16:17:22 on dist-test-slave-t3q3"
server_key: "4f635ed5e95f6a903559804ae952794a"
server_key_iv: "1011556b48f440c7e79a15d0fce9f176"
server_key_version: "encryptionkey@0"
I20260806 16:17:22.567471 19445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:22.573882 19445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:22.574841 19445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:22.575030 19445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:22.575699 19445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:22.576956 19445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:22.577008 19445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:22.577054 19445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:22.577068 19445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:22.588621 19445 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:41995
I20260806 16:17:22.588644 19626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:41995 every 8 connection(s)
I20260806 16:17:22.589918 19445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:17:22.591010 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 19445
I20260806 16:17:22.591152 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:17:22.591521 12246 external_mini_cluster.cc:1428] Setting key 654974ffc37540ba1f73aa60c3785360
I20260806 16:17:22.594430 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43023
--builtin_ntp_servers=127.11.245.148:33151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 16:17:22.599651 19419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:22.592550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:49275 (local address 127.11.245.190:43023)
0806 16:17:22.592730 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:22.592735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:22.594489 (+ 1754us) server_negotiation.cc:407] Connection header received
0806 16:17:22.597229 (+ 2740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:22.597235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:22.597313 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:22.597441 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:22.598907 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:22.598914 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:22.598921 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:17:22.598945 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:22.598987 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:17:22.599004 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:22.599132 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:22.599341 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 16:17:22.599411 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 16:17:22.600850 19631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:22.593145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43023 (local address 127.11.245.129:49275)
0806 16:17:22.594233 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:22.594344 (+ 111us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:22.596815 (+ 2471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:22.597682 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:22.597692 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:22.597815 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:22.598558 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:17:22.598560 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:22.598612 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:17:22.598700 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:22.599135 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:22.599145 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:22.599151 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 16:17:22.599361 (+ 210us) client_negotiation.cc:240] Negotiation successful
0806 16:17:22.599656 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":202,"threads_started":1}
I20260806 16:17:22.602855 19627 heartbeater.cc:347] Connected to a master server at 127.11.245.190:43023
I20260806 16:17:22.603223 19627 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:22.603834 19627 heartbeater.cc:511] Master 127.11.245.190:43023 requested a full tablet report, sending...
I20260806 16:17:22.606046 19360 ts_manager.cc:194] Registered new tserver with Master: 8a99dd9e4eb944d6927aea30b93eca31 (127.11.245.129:41995)
W20260806 16:17:22.760939 19632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:22.761390 19632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:22.761471 19632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:22.766327 19632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:22.766438 19632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:22.766541 19632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:17:22.772568 19632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:22.774366 19632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:22.776458 19632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:22.790221 19640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:22.790144 19641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:22.790244 19644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:22.790592 19632 server_base.cc:1034] running on GCE node
I20260806 16:17:22.791236 19632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:22.792285 19632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:22.793532 19632 hybrid_clock.cc:648] HybridClock initialized: now 1786033042793487 us; error 61 us; skew 500 ppm
I20260806 16:17:22.796452 19632 webserver.cc:460] Webserver started at http://127.11.245.130:36471/ using document root <none> and password file <none>
I20260806 16:17:22.797289 19632 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:22.797509 19632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:22.797890 19632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:22.801292 19632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "c911d69a2bdb44f2b624c0cdd20aeb03"
format_stamp: "Formatted at 2026-08-06 16:17:22 on dist-test-slave-t3q3"
server_key: "7777ece93ab1f7bab172815958a7a4f0"
server_key_iv: "63c16260d0f0debc52a46d6f2f6c059e"
server_key_version: "encryptionkey@0"
I20260806 16:17:22.802101 19632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "c911d69a2bdb44f2b624c0cdd20aeb03"
format_stamp: "Formatted at 2026-08-06 16:17:22 on dist-test-slave-t3q3"
server_key: "7777ece93ab1f7bab172815958a7a4f0"
server_key_iv: "63c16260d0f0debc52a46d6f2f6c059e"
server_key_version: "encryptionkey@0"
I20260806 16:17:22.808033 19632 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 16:17:22.812041 19650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:22.813997 19632 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260806 16:17:22.814213 19632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "c911d69a2bdb44f2b624c0cdd20aeb03"
format_stamp: "Formatted at 2026-08-06 16:17:22 on dist-test-slave-t3q3"
server_key: "7777ece93ab1f7bab172815958a7a4f0"
server_key_iv: "63c16260d0f0debc52a46d6f2f6c059e"
server_key_version: "encryptionkey@0"
I20260806 16:17:22.814409 19632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:22.821344 19632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:22.822289 19632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:22.822543 19632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:22.823345 19632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:22.824812 19632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:22.824898 19632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:22.824985 19632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:22.825031 19632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:22.836771 19632 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:40411
I20260806 16:17:22.836796 19815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:40411 every 8 connection(s)
I20260806 16:17:22.838233 19632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:17:22.847218 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 19632
I20260806 16:17:22.847394 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:17:22.847692 19419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:22.841252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:37763 (local address 127.11.245.190:43023)
0806 16:17:22.841462 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:22.841468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:22.843009 (+ 1541us) server_negotiation.cc:407] Connection header received
0806 16:17:22.844688 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:22.844693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:22.844747 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:22.844841 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:22.846964 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:22.846968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:22.846971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:17:22.846989 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:22.847019 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:17:22.847029 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:22.847133 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:22.847437 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 16:17:22.847525 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 16:17:22.847846 12246 external_mini_cluster.cc:1428] Setting key 5d5dc6c3109bdd909b58ab73728d8eda
I20260806 16:17:22.849757 19818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:22.841846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43023 (local address 127.11.245.130:37763)
0806 16:17:22.842801 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:22.842866 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:22.844411 (+ 1545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:22.845146 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:22.845155 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:22.845273 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:22.846376 (+ 1103us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:17:22.846380 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:22.846456 (+ 76us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:17:22.846608 (+ 152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:22.847152 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:22.847170 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:22.847179 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 16:17:22.847463 (+ 284us) client_negotiation.cc:240] Negotiation successful
0806 16:17:22.848101 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":152,"threads_started":1}
I20260806 16:17:22.851032 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43023
--builtin_ntp_servers=127.11.245.148:33151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 16:17:22.851188 19816 heartbeater.cc:347] Connected to a master server at 127.11.245.190:43023
I20260806 16:17:22.851454 19816 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:22.852041 19816 heartbeater.cc:511] Master 127.11.245.190:43023 requested a full tablet report, sending...
I20260806 16:17:22.853632 19360 ts_manager.cc:194] Registered new tserver with Master: c911d69a2bdb44f2b624c0cdd20aeb03 (127.11.245.130:40411)
W20260806 16:17:22.996493 19819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:22.996906 19819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:22.996989 19819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:23.001266 19819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:23.001382 19819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:23.001490 19819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:17:23.006649 19819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:23.008292 19819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:23.010160 19819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:23.023203 19834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:23.023221 19836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:23.023203 19830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:23.023599 19819 server_base.cc:1034] running on GCE node
I20260806 16:17:23.024386 19819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:23.025249 19819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:23.026453 19819 hybrid_clock.cc:648] HybridClock initialized: now 1786033043026434 us; error 34 us; skew 500 ppm
I20260806 16:17:23.028929 19819 webserver.cc:460] Webserver started at http://127.11.245.131:38277/ using document root <none> and password file <none>
I20260806 16:17:23.029667 19819 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:23.029735 19819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:23.029958 19819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:23.032730 19819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "ec2c2e30c2fd4130b26ee48c9c941d37"
format_stamp: "Formatted at 2026-08-06 16:17:23 on dist-test-slave-t3q3"
server_key: "0801f922e6d60a898bb0508affe0fc0b"
server_key_iv: "f1db76ce170c964f1a02887bbf60039a"
server_key_version: "encryptionkey@0"
I20260806 16:17:23.033422 19819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "ec2c2e30c2fd4130b26ee48c9c941d37"
format_stamp: "Formatted at 2026-08-06 16:17:23 on dist-test-slave-t3q3"
server_key: "0801f922e6d60a898bb0508affe0fc0b"
server_key_iv: "f1db76ce170c964f1a02887bbf60039a"
server_key_version: "encryptionkey@0"
I20260806 16:17:23.037998 19819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:17:23.041401 19846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:23.043092 19819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:17:23.043253 19819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "ec2c2e30c2fd4130b26ee48c9c941d37"
format_stamp: "Formatted at 2026-08-06 16:17:23 on dist-test-slave-t3q3"
server_key: "0801f922e6d60a898bb0508affe0fc0b"
server_key_iv: "f1db76ce170c964f1a02887bbf60039a"
server_key_version: "encryptionkey@0"
I20260806 16:17:23.043413 19819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:23.049021 19819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:23.049839 19819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:23.050060 19819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:23.050773 19819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:23.051975 19819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:23.052026 19819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:23.052093 19819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:23.052155 19819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:23.063803 19819 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:39147
I20260806 16:17:23.063839 20015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:39147 every 8 connection(s)
I20260806 16:17:23.065145 19819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:17:23.072237 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 19819
I20260806 16:17:23.072748 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:17:23.073207 12246 external_mini_cluster.cc:1428] Setting key 222bd308ccfc20a3a19a7aa0d5cad621
I20260806 16:17:23.073774 19419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.068039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:42087 (local address 127.11.245.190:43023)
0806 16:17:23.068273 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:23.068279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:23.069745 (+ 1466us) server_negotiation.cc:407] Connection header received
0806 16:17:23.071364 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:23.071367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:23.071422 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:23.071517 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:23.073134 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:23.073140 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:23.073143 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:17:23.073160 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:23.073205 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:17:23.073221 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:23.073348 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:23.073538 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 16:17:23.073596 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 16:17:23.075156 20018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.068630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43023 (local address 127.11.245.131:42087)
0806 16:17:23.069535 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:23.069589 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:23.071091 (+ 1502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:23.071731 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:23.071738 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:23.071858 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:23.072725 (+ 867us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:17:23.072728 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:23.072779 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:17:23.072896 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:23.073336 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:23.073347 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:23.073354 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 16:17:23.073573 (+ 219us) client_negotiation.cc:240] Negotiation successful
0806 16:17:23.073902 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":169,"threads_started":1}
I20260806 16:17:23.076575 20016 heartbeater.cc:347] Connected to a master server at 127.11.245.190:43023
I20260806 16:17:23.076874 20016 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:23.077437 20016 heartbeater.cc:511] Master 127.11.245.190:43023 requested a full tablet report, sending...
I20260806 16:17:23.078717 19360 ts_manager.cc:194] Registered new tserver with Master: ec2c2e30c2fd4130b26ee48c9c941d37 (127.11.245.131:39147)
I20260806 16:17:23.088804 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:17:23.093767 19419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.091645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44024 (local address 127.11.245.190:43023)
0806 16:17:23.091864 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:23.091870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:23.092004 (+ 134us) server_negotiation.cc:407] Connection header received
0806 16:17:23.092200 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:23.092203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:23.092250 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:23.092343 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:23.093218 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:23.093222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:23.093224 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:17:23.093245 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:23.093269 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:17:23.093281 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:23.093371 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:23.093476 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 16:17:23.093551 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 16:17:23.097761 19360 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:17:23.101972 19360 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:17:23.118036 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.114580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:39147 (local address 127.0.0.1:34568)
0806 16:17:23.115169 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:23.115182 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:23.115263 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:23.116393 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:23.116399 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:23.116432 (+ 33us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:23.116907 (+ 475us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:17:23.116912 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:23.116940 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:17:23.117043 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:23.117622 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:23.117627 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:23.117632 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 16:17:23.117722 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 16:17:23.117825 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":115,"threads_started":1}
I20260806 16:17:23.118036 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.113603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:40411 (local address 127.0.0.1:43572)
0806 16:17:23.114862 (+ 1259us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:23.114876 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:23.114962 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:23.115824 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:23.115831 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:23.116113 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:23.117232 (+ 1119us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:17:23.117235 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:23.117248 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:17:23.117279 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:23.117658 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:23.117661 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:23.117664 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 16:17:23.117722 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 16:17:23.117834 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1189,"mutex_wait_us":169,"thread_start_us":144,"threads_started":1}
I20260806 16:17:23.118825 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.115304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34568 (local address 127.11.245.131:39147)
0806 16:17:23.115740 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:23.115754 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:23.115782 (+ 28us) server_negotiation.cc:407] Connection header received
0806 16:17:23.115841 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:23.115846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:23.116199 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:23.116656 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:23.117368 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:23.117378 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:23.117394 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:17:23.117407 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:23.117469 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:17:23.117501 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:23.118452 (+ 951us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:23.118556 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 16:17:23.118648 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":209,"threads_started":1}
I20260806 16:17:23.118888 20034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.113959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43572 (local address 127.11.245.130:40411)
0806 16:17:23.115031 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:23.115045 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:23.115070 (+ 25us) server_negotiation.cc:407] Connection header received
0806 16:17:23.115127 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:23.115131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:23.115400 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:23.115799 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:23.117438 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:23.117443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:23.117453 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:17:23.117469 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:23.117519 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:17:23.117554 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:23.118577 (+ 1023us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:23.118635 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 16:17:23.118704 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":957,"thread_start_us":139,"threads_started":1}
I20260806 16:17:23.119144 20031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.113607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:41995 (local address 127.0.0.1:35186)
0806 16:17:23.114457 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:23.114523 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:23.114667 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:23.116737 (+ 2070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:23.116762 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:23.116785 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:23.117212 (+ 427us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:17:23.117214 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:23.117226 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:17:23.117255 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:23.117592 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:23.117603 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:23.117610 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 16:17:23.117737 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 16:17:23.117818 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":1,"thread_start_us":135,"threads_started":1}
I20260806 16:17:23.119206 20033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.113837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35186 (local address 127.11.245.129:41995)
0806 16:17:23.114800 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:23.114810 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:23.114847 (+ 37us) server_negotiation.cc:407] Connection header received
0806 16:17:23.114936 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:23.114944 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:23.115908 (+ 964us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:23.116098 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:23.117401 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:23.117406 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:23.117416 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:17:23.117428 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:23.117466 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:17:23.117498 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:23.117635 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:23.117920 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 16:17:23.118027 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":827,"thread_start_us":125,"threads_started":1}
I20260806 16:17:23.122891 19723 tablet_service.cc:1467] Processing CreateTablet for tablet e1ebdcf1543e4164956fb0c875682c78 (DEFAULT_TABLE table=test-table [id=07a7f27f641f404e8cf1bdaad2fdd605]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:23.122893 19917 tablet_service.cc:1467] Processing CreateTablet for tablet e1ebdcf1543e4164956fb0c875682c78 (DEFAULT_TABLE table=test-table [id=07a7f27f641f404e8cf1bdaad2fdd605]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:23.122890 19532 tablet_service.cc:1467] Processing CreateTablet for tablet e1ebdcf1543e4164956fb0c875682c78 (DEFAULT_TABLE table=test-table [id=07a7f27f641f404e8cf1bdaad2fdd605]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:23.124488 19917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1ebdcf1543e4164956fb0c875682c78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:23.124487 19532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1ebdcf1543e4164956fb0c875682c78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:23.124491 19723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1ebdcf1543e4164956fb0c875682c78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:23.133363 20037 tablet_bootstrap.cc:492] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31: Bootstrap starting.
I20260806 16:17:23.134018 20042 tablet_bootstrap.cc:492] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03: Bootstrap starting.
I20260806 16:17:23.134626 20039 tablet_bootstrap.cc:492] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37: Bootstrap starting.
I20260806 16:17:23.136977 20037 tablet_bootstrap.cc:654] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:23.137007 20042 tablet_bootstrap.cc:654] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:23.137006 20039 tablet_bootstrap.cc:654] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:23.138019 20039 log.cc:824] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:23.138019 20042 log.cc:824] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:23.138018 20037 log.cc:824] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:23.140868 20039 tablet_bootstrap.cc:492] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37: No bootstrap required, opened a new log
I20260806 16:17:23.140867 20037 tablet_bootstrap.cc:492] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31: No bootstrap required, opened a new log
I20260806 16:17:23.141124 20037 ts_tablet_manager.cc:1397] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 16:17:23.141124 20039 ts_tablet_manager.cc:1397] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 16:17:23.141443 20042 tablet_bootstrap.cc:492] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03: No bootstrap required, opened a new log
I20260806 16:17:23.141672 20042 ts_tablet_manager.cc:1397] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 16:17:23.145959 20042 raft_consensus.cc:348] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a99dd9e4eb944d6927aea30b93eca31" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41995 } } peers { permanent_uuid: "c911d69a2bdb44f2b624c0cdd20aeb03" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40411 } } peers { permanent_uuid: "ec2c2e30c2fd4130b26ee48c9c941d37" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39147 } }
I20260806 16:17:23.146400 20042 raft_consensus.cc:374] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:23.146473 20042 raft_consensus.cc:729] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c911d69a2bdb44f2b624c0cdd20aeb03, State: Initialized, Role: FOLLOWER
I20260806 16:17:23.146347 20037 raft_consensus.cc:348] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a99dd9e4eb944d6927aea30b93eca31" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41995 } } peers { permanent_uuid: "c911d69a2bdb44f2b624c0cdd20aeb03" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40411 } } peers { permanent_uuid: "ec2c2e30c2fd4130b26ee48c9c941d37" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39147 } }
I20260806 16:17:23.146664 20037 raft_consensus.cc:374] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:23.146353 20039 raft_consensus.cc:348] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a99dd9e4eb944d6927aea30b93eca31" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41995 } } peers { permanent_uuid: "c911d69a2bdb44f2b624c0cdd20aeb03" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40411 } } peers { permanent_uuid: "ec2c2e30c2fd4130b26ee48c9c941d37" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39147 } }
I20260806 16:17:23.146728 20037 raft_consensus.cc:729] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a99dd9e4eb944d6927aea30b93eca31, State: Initialized, Role: FOLLOWER
I20260806 16:17:23.146847 20039 raft_consensus.cc:374] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:23.146906 20039 raft_consensus.cc:729] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec2c2e30c2fd4130b26ee48c9c941d37, State: Initialized, Role: FOLLOWER
I20260806 16:17:23.147193 20042 consensus_queue.cc:260] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a99dd9e4eb944d6927aea30b93eca31" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41995 } } peers { permanent_uuid: "c911d69a2bdb44f2b624c0cdd20aeb03" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40411 } } peers { permanent_uuid: "ec2c2e30c2fd4130b26ee48c9c941d37" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39147 } }
I20260806 16:17:23.147322 20039 consensus_queue.cc:260] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a99dd9e4eb944d6927aea30b93eca31" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41995 } } peers { permanent_uuid: "c911d69a2bdb44f2b624c0cdd20aeb03" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40411 } } peers { permanent_uuid: "ec2c2e30c2fd4130b26ee48c9c941d37" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39147 } }
I20260806 16:17:23.147315 20037 consensus_queue.cc:260] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a99dd9e4eb944d6927aea30b93eca31" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41995 } } peers { permanent_uuid: "c911d69a2bdb44f2b624c0cdd20aeb03" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40411 } } peers { permanent_uuid: "ec2c2e30c2fd4130b26ee48c9c941d37" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39147 } }
I20260806 16:17:23.148113 19816 heartbeater.cc:503] Master 127.11.245.190:43023 was elected leader, sending a full tablet report...
I20260806 16:17:23.148561 20037 ts_tablet_manager.cc:1428] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 16:17:23.148561 20039 ts_tablet_manager.cc:1428] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 16:17:23.148559 20042 ts_tablet_manager.cc:1428] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 16:17:23.148931 19627 heartbeater.cc:503] Master 127.11.245.190:43023 was elected leader, sending a full tablet report...
I20260806 16:17:23.148931 20016 heartbeater.cc:503] Master 127.11.245.190:43023 was elected leader, sending a full tablet report...
W20260806 16:17:23.316607 20017 tablet.cc:2369] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:17:23.319602 20049 raft_consensus.cc:482] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:23.319942 20049 raft_consensus.cc:504] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a99dd9e4eb944d6927aea30b93eca31" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41995 } } peers { permanent_uuid: "c911d69a2bdb44f2b624c0cdd20aeb03" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40411 } } peers { permanent_uuid: "ec2c2e30c2fd4130b26ee48c9c941d37" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39147 } }
I20260806 16:17:23.321712 20049 leader_election.cc:288] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c911d69a2bdb44f2b624c0cdd20aeb03 (127.11.245.130:40411), ec2c2e30c2fd4130b26ee48c9c941d37 (127.11.245.131:39147)
I20260806 16:17:23.324702 20052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.322135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:39147 (local address 127.11.245.129:36605)
0806 16:17:23.322736 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:23.322751 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:23.322882 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:23.323294 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:23.323297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:23.323315 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:23.323969 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:17:23.323974 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:23.323992 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:17:23.324029 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:23.324353 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:23.324357 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:23.324361 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 16:17:23.324459 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 16:17:23.324505 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":246,"threads_started":1}
I20260806 16:17:23.324685 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.322330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:36605 (local address 127.11.245.131:39147)
0806 16:17:23.322551 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:23.322557 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:23.322863 (+ 306us) server_negotiation.cc:407] Connection header received
0806 16:17:23.323000 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:23.323004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:23.323066 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:23.323165 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:23.324210 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:23.324214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:23.324216 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:17:23.324228 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:23.324253 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:17:23.324261 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:23.324376 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:23.324451 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 16:17:23.324507 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 16:17:23.324906 20034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.322484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:45513 (local address 127.11.245.130:40411)
0806 16:17:23.322676 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:23.322681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:23.323019 (+ 338us) server_negotiation.cc:407] Connection header received
0806 16:17:23.323111 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:23.323113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:23.323286 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:23.323385 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:23.324441 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:23.324444 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:23.324446 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:17:23.324459 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:23.324482 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:17:23.324490 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:23.324610 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:23.324686 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 16:17:23.324735 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 16:17:23.324885 20051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.322135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:40411 (local address 127.11.245.129:45513)
0806 16:17:23.322871 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:23.322889 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:23.323009 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:23.323448 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:23.323451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:23.323465 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:23.324202 (+ 737us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:17:23.324204 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:23.324214 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:17:23.324251 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:23.324588 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:23.324592 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:23.324595 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 16:17:23.324678 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 16:17:23.324722 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":112,"thread_start_us":136,"threads_started":1}
I20260806 16:17:23.325990 19751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1ebdcf1543e4164956fb0c875682c78" candidate_uuid: "8a99dd9e4eb944d6927aea30b93eca31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c911d69a2bdb44f2b624c0cdd20aeb03" is_pre_election: true
I20260806 16:17:23.325990 19946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1ebdcf1543e4164956fb0c875682c78" candidate_uuid: "8a99dd9e4eb944d6927aea30b93eca31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec2c2e30c2fd4130b26ee48c9c941d37" is_pre_election: true
I20260806 16:17:23.326396 19946 raft_consensus.cc:2454] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a99dd9e4eb944d6927aea30b93eca31 in term 0.
I20260806 16:17:23.326396 19751 raft_consensus.cc:2454] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a99dd9e4eb944d6927aea30b93eca31 in term 0.
I20260806 16:17:23.326990 19465 leader_election.cc:302] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8a99dd9e4eb944d6927aea30b93eca31, c911d69a2bdb44f2b624c0cdd20aeb03, ec2c2e30c2fd4130b26ee48c9c941d37; no voters:
I20260806 16:17:23.327378 20049 raft_consensus.cc:2781] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:23.327476 20049 raft_consensus.cc:482] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:23.327536 20049 raft_consensus.cc:3037] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:23.329208 20049 raft_consensus.cc:504] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a99dd9e4eb944d6927aea30b93eca31" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41995 } } peers { permanent_uuid: "c911d69a2bdb44f2b624c0cdd20aeb03" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40411 } } peers { permanent_uuid: "ec2c2e30c2fd4130b26ee48c9c941d37" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39147 } }
I20260806 16:17:23.329744 20049 leader_election.cc:288] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [CANDIDATE]: Term 1 election: Requested vote from peers c911d69a2bdb44f2b624c0cdd20aeb03 (127.11.245.130:40411), ec2c2e30c2fd4130b26ee48c9c941d37 (127.11.245.131:39147)
I20260806 16:17:23.330269 19751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1ebdcf1543e4164956fb0c875682c78" candidate_uuid: "8a99dd9e4eb944d6927aea30b93eca31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c911d69a2bdb44f2b624c0cdd20aeb03"
I20260806 16:17:23.330228 19946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1ebdcf1543e4164956fb0c875682c78" candidate_uuid: "8a99dd9e4eb944d6927aea30b93eca31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec2c2e30c2fd4130b26ee48c9c941d37"
I20260806 16:17:23.330431 19751 raft_consensus.cc:3037] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:23.330593 19946 raft_consensus.cc:3037] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:23.332142 19751 raft_consensus.cc:2454] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a99dd9e4eb944d6927aea30b93eca31 in term 1.
I20260806 16:17:23.332329 19946 raft_consensus.cc:2454] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a99dd9e4eb944d6927aea30b93eca31 in term 1.
I20260806 16:17:23.332630 19465 leader_election.cc:302] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a99dd9e4eb944d6927aea30b93eca31, c911d69a2bdb44f2b624c0cdd20aeb03; no voters:
I20260806 16:17:23.332856 20049 raft_consensus.cc:2781] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:23.333197 20049 raft_consensus.cc:686] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [term 1 LEADER]: Becoming Leader. State: Replica: 8a99dd9e4eb944d6927aea30b93eca31, State: Running, Role: LEADER
I20260806 16:17:23.333654 20049 consensus_queue.cc:237] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a99dd9e4eb944d6927aea30b93eca31" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41995 } } peers { permanent_uuid: "c911d69a2bdb44f2b624c0cdd20aeb03" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40411 } } peers { permanent_uuid: "ec2c2e30c2fd4130b26ee48c9c941d37" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39147 } }
I20260806 16:17:23.338239 19360 catalog_manager.cc:5495] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a99dd9e4eb944d6927aea30b93eca31 (127.11.245.129). New cstate: current_term: 1 leader_uuid: "8a99dd9e4eb944d6927aea30b93eca31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a99dd9e4eb944d6927aea30b93eca31" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c911d69a2bdb44f2b624c0cdd20aeb03" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec2c2e30c2fd4130b26ee48c9c941d37" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39147 } health_report { overall_health: UNKNOWN } } }
W20260806 16:17:23.340327 19817 tablet.cc:2369] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:17:23.341706 19630 tablet.cc:2369] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:17:23.362743 20033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.360628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35190 (local address 127.11.245.129:41995)
0806 16:17:23.360849 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:23.360855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:23.360882 (+ 27us) server_negotiation.cc:407] Connection header received
0806 16:17:23.361104 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:23.361107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:23.361194 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:23.361300 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:23.362213 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:23.362217 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:23.362220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:17:23.362233 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:23.362258 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:17:23.362267 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:23.362409 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:23.362507 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 16:17:23.362558 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 16:17:23.370764 19751 raft_consensus.cc:1261] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03 [term 1 FOLLOWER]: Refusing update from remote peer 8a99dd9e4eb944d6927aea30b93eca31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:23.370765 19946 raft_consensus.cc:1261] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37 [term 1 FOLLOWER]: Refusing update from remote peer 8a99dd9e4eb944d6927aea30b93eca31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:23.371730 20049 consensus_queue.cc:1035] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c911d69a2bdb44f2b624c0cdd20aeb03" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:23.372025 20053 consensus_queue.cc:1035] T e1ebdcf1543e4164956fb0c875682c78 P 8a99dd9e4eb944d6927aea30b93eca31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec2c2e30c2fd4130b26ee48c9c941d37" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:23.380353 20058 mvcc.cc:205] Tried to move back new op lower bound from 7315591345637072896 to 7315591345498103808. Current Snapshot: MvccSnapshot[applied={T|T < 7315591345637072896}]
I20260806 16:17:23.392975 19359 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:23.393265 19359 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:23.396696 19359 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd2eb47c950f4c01af0402a78a2f767d: Sending DeleteTablet for 3 replicas of tablet e1ebdcf1543e4164956fb0c875682c78
I20260806 16:17:23.397954 19532 tablet_service.cc:1514] Processing DeleteTablet for tablet e1ebdcf1543e4164956fb0c875682c78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:23 UTC) from {username='slave'} at 127.0.0.1:35186
I20260806 16:17:23.397943 19723 tablet_service.cc:1514] Processing DeleteTablet for tablet e1ebdcf1543e4164956fb0c875682c78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:23 UTC) from {username='slave'} at 127.0.0.1:43572
I20260806 16:17:23.398067 19917 tablet_service.cc:1514] Processing DeleteTablet for tablet e1ebdcf1543e4164956fb0c875682c78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:23 UTC) from {username='slave'} at 127.0.0.1:34568
I20260806 16:17:23.398679 20071 tablet_replica.cc:330] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03: stopping tablet replica
I20260806 16:17:23.399019 20074 tablet_replica.cc:330] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37: stopping tablet replica
I20260806 16:17:23.399183 20071 raft_consensus.cc:2229] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:23.399468 20074 raft_consensus.cc:2229] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:23.399801 20074 raft_consensus.cc:2258] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:23.399808 20071 raft_consensus.cc:2258] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:23.400311 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 19445
I20260806 16:17:23.402009 20071 ts_tablet_manager.cc:1905] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:23.405514 20071 ts_tablet_manager.cc:1918] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:23.405639 20071 log.cc:1196] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/e1ebdcf1543e4164956fb0c875682c78
I20260806 16:17:23.406054 20071 ts_tablet_manager.cc:1939] T e1ebdcf1543e4164956fb0c875682c78 P c911d69a2bdb44f2b624c0cdd20aeb03: Deleting consensus metadata
I20260806 16:17:23.408181 19337 catalog_manager.cc:4841] TS c911d69a2bdb44f2b624c0cdd20aeb03 (127.11.245.130:40411): tablet e1ebdcf1543e4164956fb0c875682c78 (table test-table [id=07a7f27f641f404e8cf1bdaad2fdd605]) successfully deleted
I20260806 16:17:23.410040 20074 ts_tablet_manager.cc:1905] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:23.414273 20074 ts_tablet_manager.cc:1918] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:23.414427 20074 log.cc:1196] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/e1ebdcf1543e4164956fb0c875682c78
I20260806 16:17:23.414872 20074 ts_tablet_manager.cc:1939] T e1ebdcf1543e4164956fb0c875682c78 P ec2c2e30c2fd4130b26ee48c9c941d37: Deleting consensus metadata
I20260806 16:17:23.416960 19339 catalog_manager.cc:4841] TS ec2c2e30c2fd4130b26ee48c9c941d37 (127.11.245.131:39147): tablet e1ebdcf1543e4164956fb0c875682c78 (table test-table [id=07a7f27f641f404e8cf1bdaad2fdd605]) successfully deleted
I20260806 16:17:23.419209 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 19632
W20260806 16:17:23.419567 19339 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:41995 (error 108)
I20260806 16:17:23.430567 20031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.430117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:41995 (local address 127.0.0.1:35198)
0806 16:17:23.430343 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:23.430443 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:41995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260806 16:17:23.430836 19339 catalog_manager.cc:4584] TS 8a99dd9e4eb944d6927aea30b93eca31 (127.11.245.129:41995): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e1ebdcf1543e4164956fb0c875682c78: Network error: Client connection negotiation failed: client connection to 127.11.245.129:41995: connect: Connection refused (error 111)
I20260806 16:17:23.438360 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 19819
I20260806 16:17:23.457051 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 19310
2026-08-06T16:17:23Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1431 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 16:17:23.493937 12246 mini_oidc.cc:152] Starting JWKS server
I20260806 16:17:23.495968 12246 webserver.cc:460] Webserver started at https://127.0.0.1:39711/ using document root <none> and password file <none>
I20260806 16:17:23.496299 12246 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 16:17:23.496630 12246 webserver.cc:460] Webserver started at http://127.0.0.1:44973/ using document root <none> and password file <none>
2026-08-06T16:17:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:23Z Disabled control of system clock
I20260806 16:17:23.510946 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:38457
--webserver_interface=127.11.245.190
--webserver_port=0
--builtin_ntp_servers=127.11.245.148:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:38457
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39711/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/testchainca.cert with env {}
W20260806 16:17:23.653971 20084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:23.654704 20084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:23.654793 20084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:23.659838 20084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:23.659929 20084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:23.659945 20084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:23.659963 20084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:23.659979 20084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 16:17:23.659999 20084 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 16:17:23.660013 20084 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39711/jwks.json
I20260806 16:17:23.666801 20084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:38457
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:38457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39711/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:23.668453 20084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:23.669853 20084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:23.680934 20092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:23.681022 20095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:23.680987 20084 server_base.cc:1034] running on GCE node
W20260806 16:17:23.680936 20091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:23.682106 20084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:23.683991 20084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:23.685397 20084 hybrid_clock.cc:648] HybridClock initialized: now 1786033043685340 us; error 72 us; skew 500 ppm
I20260806 16:17:23.688418 20084 webserver.cc:460] Webserver started at http://127.11.245.190:38643/ using document root <none> and password file <none>
I20260806 16:17:23.689240 20084 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:23.689352 20084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:23.689648 20084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:23.692445 20084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "c9c0ccbe4d2f4516a74af91f85fb8003"
format_stamp: "Formatted at 2026-08-06 16:17:23 on dist-test-slave-t3q3"
server_key: "dfde8a00582d8f7834eb3bb6c132a59e"
server_key_iv: "399ade450d9353c7e7db6a2e02bd2874"
server_key_version: "encryptionkey@0"
I20260806 16:17:23.693223 20084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "c9c0ccbe4d2f4516a74af91f85fb8003"
format_stamp: "Formatted at 2026-08-06 16:17:23 on dist-test-slave-t3q3"
server_key: "dfde8a00582d8f7834eb3bb6c132a59e"
server_key_iv: "399ade450d9353c7e7db6a2e02bd2874"
server_key_version: "encryptionkey@0"
I20260806 16:17:23.698189 20084 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:17:23.701696 20108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:23.703299 20084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:17:23.703480 20084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "c9c0ccbe4d2f4516a74af91f85fb8003"
format_stamp: "Formatted at 2026-08-06 16:17:23 on dist-test-slave-t3q3"
server_key: "dfde8a00582d8f7834eb3bb6c132a59e"
server_key_iv: "399ade450d9353c7e7db6a2e02bd2874"
server_key_version: "encryptionkey@0"
I20260806 16:17:23.703632 20084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:23.720599 20084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:23.721594 20084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:23.721853 20084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:23.731513 20084 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:38457
I20260806 16:17:23.731547 20182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:38457 every 8 connection(s)
I20260806 16:17:23.732877 20084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:17:23.737649 20183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:23.740435 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 20084
I20260806 16:17:23.740604 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:17:23.740986 12246 external_mini_cluster.cc:1428] Setting key f5f4a02a7207a5521ec1119ceb188fb4
I20260806 16:17:23.746071 20183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003: Bootstrap starting.
I20260806 16:17:23.749907 20183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:23.751267 20183 log.cc:824] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:23.753376 20188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.742664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53922 (local address 127.11.245.190:38457)
0806 16:17:23.743462 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:23.743481 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:23.743527 (+ 46us) server_negotiation.cc:407] Connection header received
0806 16:17:23.744641 (+ 1114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:23.744667 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:23.745153 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:23.745496 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:23.745942 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:23.747054 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:23.748601 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:23.749424 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:23.749624 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:23.750748 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:23.750756 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:23.750779 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:17:23.751488 (+ 709us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:23.751580 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:17:23.751615 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:23.751745 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:23.751928 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 16:17:23.752331 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":201,"threads_started":1}
I20260806 16:17:23.754292 20183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003: No bootstrap required, opened a new log
I20260806 16:17:23.757325 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:17:23.759081 20183 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c0ccbe4d2f4516a74af91f85fb8003" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 38457 } }
I20260806 16:17:23.759430 20183 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:23.759516 20183 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9c0ccbe4d2f4516a74af91f85fb8003, State: Initialized, Role: FOLLOWER
I20260806 16:17:23.760033 20183 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c0ccbe4d2f4516a74af91f85fb8003" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 38457 } }
I20260806 16:17:23.760269 20183 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:17:23.760346 20183 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:17:23.760476 20183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:23.761811 20183 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c0ccbe4d2f4516a74af91f85fb8003" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 38457 } }
I20260806 16:17:23.762228 20183 leader_election.cc:302] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9c0ccbe4d2f4516a74af91f85fb8003; no voters:
I20260806 16:17:23.762635 20183 leader_election.cc:288] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:17:23.762831 20192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:23.763111 20192 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [term 1 LEADER]: Becoming Leader. State: Replica: c9c0ccbe4d2f4516a74af91f85fb8003, State: Running, Role: LEADER
I20260806 16:17:23.763478 20192 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c0ccbe4d2f4516a74af91f85fb8003" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 38457 } }
I20260806 16:17:23.764216 20183 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:23.765615 20193 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9c0ccbe4d2f4516a74af91f85fb8003" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c0ccbe4d2f4516a74af91f85fb8003" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 38457 } } }
I20260806 16:17:23.765658 20194 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9c0ccbe4d2f4516a74af91f85fb8003. Latest consensus state: current_term: 1 leader_uuid: "c9c0ccbe4d2f4516a74af91f85fb8003" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c0ccbe4d2f4516a74af91f85fb8003" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 38457 } } }
I20260806 16:17:23.765803 20193 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:23.765806 20194 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003 [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:23.766279 20204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:23.770298 20204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:23.777830 20204 catalog_manager.cc:1347] Generated new cluster ID: db66d7332a434fec861161c900e663ab
I20260806 16:17:23.777940 20204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:23.788640 20204 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:17:23.789915 20204 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:23.809548 20204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9c0ccbe4d2f4516a74af91f85fb8003: Generated new TSK 0
I20260806 16:17:23.810776 20204 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 16:17:23.845595 20230 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 16:17:23.845844 20230 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 16:17:23.846621 20188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:23.826882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53934 (local address 127.11.245.190:38457)
0806 16:17:23.827056 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:23.827062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:23.827272 (+ 210us) server_negotiation.cc:407] Connection header received
0806 16:17:23.827530 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:23.827534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:23.827614 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:23.827743 (+ 129us) server_negotiation.cc:226] Negotiated authn=JWT
0806 16:17:23.828013 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:23.828978 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:23.830018 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:23.830421 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:23.830536 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:23.831234 (+ 698us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 16:17:23.845993 (+ 14759us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39711/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 16:17:23.846405 (+ 412us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53934: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39711/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":75}
W20260806 16:17:23.846737 20188 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53934: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39711/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 16:17:23.848284 20229 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:23.826744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:38457 (local address 127.0.0.1:53934)
0806 16:17:23.827166 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:23.827181 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:23.827327 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:23.827765 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:23.827767 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:23.827775 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 16:17:23.827900 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:23.827905 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:23.829161 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:23.829165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:23.829881 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:23.829891 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:23.830603 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:23.830609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:23.830757 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:23.830895 (+ 138us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 16:17:23.846328 (+ 15433us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39711/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 16:17:23.846663 (+ 335us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.245.190:38457: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39711/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":146,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.245.190:38457: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39711/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 16:17:23.849326 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 20084
2026-08-06T16:17:23Z chronyd exiting
I20260806 16:17:23.902000 12246 test_util.cc:175] -----------------------------------------------
I20260806 16:17:23.902161 12246 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786033011181791-12246-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (410 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 16:17:23.903700 12246 mini_oidc.cc:152] Starting JWKS server
I20260806 16:17:23.905129 12246 webserver.cc:460] Webserver started at https://127.0.0.1:33431/ using document root <none> and password file <none>
I20260806 16:17:23.905314 12246 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 16:17:23.905665 12246 webserver.cc:460] Webserver started at http://127.0.0.1:42323/ using document root <none> and password file <none>
2026-08-06T16:17:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:23Z Disabled control of system clock
I20260806 16:17:23.919766 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:36867
--webserver_interface=127.11.245.190
--webserver_port=0
--builtin_ntp_servers=127.11.245.148:41047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:36867
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33431/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/testchainca.cert with env {}
W20260806 16:17:24.067370 20237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:24.067868 20237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:24.067920 20237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:24.072472 20237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:24.072583 20237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:24.072600 20237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:24.072619 20237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:24.072636 20237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 16:17:24.072655 20237 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 16:17:24.072669 20237 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33431/jwks.json
I20260806 16:17:24.079219 20237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:41047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:36867
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:36867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33431/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:24.080756 20237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:24.082058 20237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:24.093160 20245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:24.093160 20248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:24.093160 20244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:24.093511 20237 server_base.cc:1034] running on GCE node
I20260806 16:17:24.094486 20237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:24.096015 20237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:24.097254 20237 hybrid_clock.cc:648] HybridClock initialized: now 1786033044097232 us; error 37 us; skew 500 ppm
I20260806 16:17:24.100016 20237 webserver.cc:460] Webserver started at http://127.11.245.190:41477/ using document root <none> and password file <none>
I20260806 16:17:24.100839 20237 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:24.100945 20237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:24.101236 20237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:24.103900 20237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "b4c81b91fab94089b442e919e924628a"
format_stamp: "Formatted at 2026-08-06 16:17:24 on dist-test-slave-t3q3"
server_key: "9968d0e3c3718a3702b34a57efd41bec"
server_key_iv: "bdef2b8c733dbf2ee85415307702d9ea"
server_key_version: "encryptionkey@0"
I20260806 16:17:24.104681 20237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "b4c81b91fab94089b442e919e924628a"
format_stamp: "Formatted at 2026-08-06 16:17:24 on dist-test-slave-t3q3"
server_key: "9968d0e3c3718a3702b34a57efd41bec"
server_key_iv: "bdef2b8c733dbf2ee85415307702d9ea"
server_key_version: "encryptionkey@0"
I20260806 16:17:24.109844 20237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:17:24.113196 20256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:24.114835 20237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 16:17:24.115031 20237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "b4c81b91fab94089b442e919e924628a"
format_stamp: "Formatted at 2026-08-06 16:17:24 on dist-test-slave-t3q3"
server_key: "9968d0e3c3718a3702b34a57efd41bec"
server_key_iv: "bdef2b8c733dbf2ee85415307702d9ea"
server_key_version: "encryptionkey@0"
I20260806 16:17:24.115195 20237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:24.131129 20237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:24.132381 20237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:24.132660 20237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:24.141633 20237 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:36867
I20260806 16:17:24.141647 20328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:36867 every 8 connection(s)
I20260806 16:17:24.142952 20237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:17:24.147660 20329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:24.149530 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 20237
I20260806 16:17:24.149678 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:17:24.150126 12246 external_mini_cluster.cc:1428] Setting key b342fac9e95ba01d2899607dc5fe31c6
I20260806 16:17:24.156387 20329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a: Bootstrap starting.
I20260806 16:17:24.159473 20329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:24.160543 20329 log.cc:824] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:24.162156 20332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:24.152088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60762 (local address 127.11.245.190:36867)
0806 16:17:24.152970 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:24.152990 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:24.153051 (+ 61us) server_negotiation.cc:407] Connection header received
0806 16:17:24.153857 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:24.153880 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:24.154431 (+ 551us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:24.154864 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:24.155444 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:24.156641 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:24.157641 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:24.158189 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:24.158577 (+ 388us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:24.159636 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:24.159644 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:24.159666 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:17:24.160308 (+ 642us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:24.160383 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:17:24.160408 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:24.160526 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:24.160704 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 16:17:24.161044 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":184,"threads_started":1}
I20260806 16:17:24.163435 20329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a: No bootstrap required, opened a new log
I20260806 16:17:24.166047 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:17:24.168601 20329 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c81b91fab94089b442e919e924628a" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36867 } }
I20260806 16:17:24.168954 20329 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:24.169054 20329 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4c81b91fab94089b442e919e924628a, State: Initialized, Role: FOLLOWER
I20260806 16:17:24.169641 20329 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c81b91fab94089b442e919e924628a" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36867 } }
I20260806 16:17:24.169832 20329 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:17:24.169929 20329 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:17:24.170053 20329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:24.171303 20329 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c81b91fab94089b442e919e924628a" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36867 } }
I20260806 16:17:24.171799 20329 leader_election.cc:302] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4c81b91fab94089b442e919e924628a; no voters:
I20260806 16:17:24.172382 20329 leader_election.cc:288] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:17:24.172533 20335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:24.172787 20335 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [term 1 LEADER]: Becoming Leader. State: Replica: b4c81b91fab94089b442e919e924628a, State: Running, Role: LEADER
I20260806 16:17:24.173192 20335 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c81b91fab94089b442e919e924628a" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36867 } }
I20260806 16:17:24.174140 20329 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:24.175375 20336 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4c81b91fab94089b442e919e924628a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c81b91fab94089b442e919e924628a" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36867 } } }
I20260806 16:17:24.175424 20337 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4c81b91fab94089b442e919e924628a. Latest consensus state: current_term: 1 leader_uuid: "b4c81b91fab94089b442e919e924628a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c81b91fab94089b442e919e924628a" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36867 } } }
I20260806 16:17:24.175539 20336 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:24.175559 20337 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:24.176149 20344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:24.181133 20344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:24.189142 20344 catalog_manager.cc:1347] Generated new cluster ID: 83dddf43da91434abaf7da196df7574f
I20260806 16:17:24.189273 20344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:24.206833 20344 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:17:24.208223 20344 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:24.216094 20344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4c81b91fab94089b442e919e924628a: Generated new TSK 0
I20260806 16:17:24.217147 20344 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 16:17:24.259593 20367 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 16:17:24.259732 20367 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 16:17:24.260628 20332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:24.235139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60764 (local address 127.11.245.190:36867)
0806 16:17:24.235327 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:24.235333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:24.235623 (+ 290us) server_negotiation.cc:407] Connection header received
0806 16:17:24.235764 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:24.235767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:24.235840 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:24.235966 (+ 126us) server_negotiation.cc:226] Negotiated authn=JWT
0806 16:17:24.236322 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:24.236971 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:24.238316 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:24.238581 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:24.238682 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:24.239124 (+ 442us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 16:17:24.260123 (+ 20999us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33431/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 16:17:24.260475 (+ 352us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60764: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33431/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 16:17:24.260726 20332 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60764: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33431/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 16:17:24.260895 20366 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:24.235017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:36867 (local address 127.0.0.1:60764)
0806 16:17:24.235425 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:24.235445 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:24.235609 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:24.236003 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:24.236006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:24.236016 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0806 16:17:24.236194 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:24.236203 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:24.237099 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:24.237102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:24.238171 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:24.238183 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:24.238718 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:24.238722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:24.238821 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:24.238858 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 16:17:24.260412 (+ 21554us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33431/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 16:17:24.260677 (+ 265us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.245.190:36867: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33431/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":143,"threads_started":1}
I20260806 16:17:24.261920 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 20237
2026-08-06T16:17:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (412 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 16:17:24.316664 12246 mini_oidc.cc:152] Starting JWKS server
I20260806 16:17:24.317812 12246 webserver.cc:460] Webserver started at https://127.0.0.1:41459/ using document root <none> and password file <none>
I20260806 16:17:24.317986 12246 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 16:17:24.318323 12246 webserver.cc:460] Webserver started at http://127.0.0.1:37129/ using document root <none> and password file <none>
2026-08-06T16:17:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:24Z Disabled control of system clock
I20260806 16:17:24.331976 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:37707
--webserver_interface=127.11.245.190
--webserver_port=0
--builtin_ntp_servers=127.11.245.148:45499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:37707
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41459/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 16:17:24.477826 20377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:24.478190 20377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:24.478235 20377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:24.482990 20377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:24.483098 20377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:24.483114 20377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:24.483134 20377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:24.483150 20377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 16:17:24.483168 20377 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 16:17:24.483217 20377 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41459/jwks.json
I20260806 16:17:24.489008 20377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:37707
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:37707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41459/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:24.490671 20377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:24.492650 20377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:24.503602 20385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:24.503602 20387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:24.503659 20383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:24.504106 20377 server_base.cc:1034] running on GCE node
I20260806 16:17:24.504909 20377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:24.506556 20377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:24.507923 20377 hybrid_clock.cc:648] HybridClock initialized: now 1786033044507888 us; error 54 us; skew 500 ppm
I20260806 16:17:24.510663 20377 webserver.cc:460] Webserver started at http://127.11.245.190:44687/ using document root <none> and password file <none>
I20260806 16:17:24.511487 20377 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:24.511603 20377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:24.511927 20377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:24.514909 20377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "09e45348c6be4e3b9466a75fe3105ef7"
format_stamp: "Formatted at 2026-08-06 16:17:24 on dist-test-slave-t3q3"
server_key: "48b91870ee2d2baa869f36e3927a3782"
server_key_iv: "6efef53d9749d96f460d5e5b97f00a89"
server_key_version: "encryptionkey@0"
I20260806 16:17:24.515702 20377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "09e45348c6be4e3b9466a75fe3105ef7"
format_stamp: "Formatted at 2026-08-06 16:17:24 on dist-test-slave-t3q3"
server_key: "48b91870ee2d2baa869f36e3927a3782"
server_key_iv: "6efef53d9749d96f460d5e5b97f00a89"
server_key_version: "encryptionkey@0"
I20260806 16:17:24.521133 20377 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 16:17:24.524470 20395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:24.526274 20377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 16:17:24.526503 20377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "09e45348c6be4e3b9466a75fe3105ef7"
format_stamp: "Formatted at 2026-08-06 16:17:24 on dist-test-slave-t3q3"
server_key: "48b91870ee2d2baa869f36e3927a3782"
server_key_iv: "6efef53d9749d96f460d5e5b97f00a89"
server_key_version: "encryptionkey@0"
I20260806 16:17:24.526678 20377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:24.540459 20377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:24.541450 20377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:24.541707 20377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:24.550907 20377 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:37707
I20260806 16:17:24.550953 20465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:37707 every 8 connection(s)
I20260806 16:17:24.552253 20377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:17:24.556720 20467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:24.561479 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 20377
I20260806 16:17:24.561650 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:17:24.562074 12246 external_mini_cluster.cc:1428] Setting key 6293325ac4070180acb51cc9b8501da8
I20260806 16:17:24.564852 20467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7: Bootstrap starting.
I20260806 16:17:24.568051 20467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:24.569386 20467 log.cc:824] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:24.572245 20467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7: No bootstrap required, opened a new log
I20260806 16:17:24.574153 20470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:24.564016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55244 (local address 127.11.245.190:37707)
0806 16:17:24.564930 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:24.564950 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:24.565020 (+ 70us) server_negotiation.cc:407] Connection header received
0806 16:17:24.565831 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:24.565852 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:24.566347 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:24.566702 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:24.567150 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:24.568407 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:24.569806 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:24.570398 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:24.570679 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:24.571766 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:24.571772 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:24.571792 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:17:24.572421 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:24.572507 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:17:24.572539 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:24.572665 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:24.572797 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 16:17:24.573176 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":211,"threads_started":1}
I20260806 16:17:24.576992 20467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09e45348c6be4e3b9466a75fe3105ef7" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37707 } }
I20260806 16:17:24.577364 20467 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:24.577450 20467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09e45348c6be4e3b9466a75fe3105ef7, State: Initialized, Role: FOLLOWER
I20260806 16:17:24.577907 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:17:24.578060 20467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09e45348c6be4e3b9466a75fe3105ef7" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37707 } }
I20260806 16:17:24.578244 20467 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:17:24.578310 20467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:17:24.578430 20467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:24.579856 20467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09e45348c6be4e3b9466a75fe3105ef7" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37707 } }
I20260806 16:17:24.580377 20467 leader_election.cc:302] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09e45348c6be4e3b9466a75fe3105ef7; no voters:
I20260806 16:17:24.580803 20467 leader_election.cc:288] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:17:24.580976 20472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:24.581256 20472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [term 1 LEADER]: Becoming Leader. State: Replica: 09e45348c6be4e3b9466a75fe3105ef7, State: Running, Role: LEADER
I20260806 16:17:24.581678 20472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09e45348c6be4e3b9466a75fe3105ef7" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37707 } }
I20260806 16:17:24.582391 20467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:24.583949 20476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09e45348c6be4e3b9466a75fe3105ef7. Latest consensus state: current_term: 1 leader_uuid: "09e45348c6be4e3b9466a75fe3105ef7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09e45348c6be4e3b9466a75fe3105ef7" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37707 } } }
I20260806 16:17:24.584020 20475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09e45348c6be4e3b9466a75fe3105ef7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09e45348c6be4e3b9466a75fe3105ef7" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37707 } } }
I20260806 16:17:24.584127 20476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:24.584165 20475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7 [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:24.584635 20484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:24.588281 20484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:24.595293 20484 catalog_manager.cc:1347] Generated new cluster ID: 2db149ecf1d14ecd921551670df35307
I20260806 16:17:24.595387 20484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:24.618355 20484 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:17:24.619685 20484 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:24.626104 20484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09e45348c6be4e3b9466a75fe3105ef7: Generated new TSK 0
I20260806 16:17:24.627017 20484 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:17:24.646354 20470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:24.640900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55246 (local address 127.11.245.190:37707)
0806 16:17:24.641057 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:24.641063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:24.641239 (+ 176us) server_negotiation.cc:407] Connection header received
0806 16:17:24.641511 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:24.641514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:24.641578 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:24.641715 (+ 137us) server_negotiation.cc:226] Negotiated authn=JWT
0806 16:17:24.642001 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:24.642605 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:24.643557 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:24.643794 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:24.643915 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:24.644361 (+ 446us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 16:17:24.645892 (+ 1531us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41459/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 16:17:24.646194 (+ 302us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55246: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41459/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 16:17:24.646467 20470 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55246: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41459/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 16:17:24.649201 20504 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:24.640795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37707 (local address 127.0.0.1:55246)
0806 16:17:24.641141 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:24.641155 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:24.641304 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:24.641706 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:24.641710 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:24.641721 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0806 16:17:24.641877 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:24.641889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:24.642728 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:24.642731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:24.643423 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:24.643431 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:24.643954 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:24.643959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:24.644099 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:24.644134 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 16:17:24.646068 (+ 1934us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41459/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 16:17:24.649021 (+ 2953us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.245.190:37707: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41459/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":2706,"thread_start_us":126,"threads_started":1}
I20260806 16:17:24.650027 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 20377
E20260806 16:17:24.668649 20505 webserver.cc:533] Webserver: sslize failed:
E20260806 16:17:24.668802 20505 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T16:17:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (390 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T16:17:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:24Z Disabled control of system clock
I20260806 16:17:24.719777 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:36909
--webserver_interface=127.11.245.190
--webserver_port=0
--builtin_ntp_servers=127.11.245.148:33947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:36909
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/testchainca.cert with env {}
W20260806 16:17:24.864027 20510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:24.864512 20510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:24.864611 20510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:24.869240 20510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:24.869369 20510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:24.869400 20510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:24.869443 20510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:24.869482 20510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 16:17:24.869522 20510 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 16:17:24.869551 20510 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 16:17:24.877614 20510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:36909
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:36909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:24.879340 20510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:24.880785 20510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:24.891508 20518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:24.891522 20522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:24.891505 20519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:24.891836 20510 server_base.cc:1034] running on GCE node
I20260806 16:17:24.892796 20510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:24.894421 20510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:24.895637 20510 hybrid_clock.cc:648] HybridClock initialized: now 1786033044895617 us; error 38 us; skew 500 ppm
I20260806 16:17:24.898209 20510 webserver.cc:460] Webserver started at http://127.11.245.190:34801/ using document root <none> and password file <none>
I20260806 16:17:24.898986 20510 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:24.899089 20510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:24.899380 20510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:24.901983 20510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "d99db8b02533404fb487a2355255ec01"
format_stamp: "Formatted at 2026-08-06 16:17:24 on dist-test-slave-t3q3"
server_key: "681f441dac20ac3064773a4ffd985a5e"
server_key_iv: "60aed10ab7f096d4fb4b335b0aeaf023"
server_key_version: "encryptionkey@0"
I20260806 16:17:24.902689 20510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "d99db8b02533404fb487a2355255ec01"
format_stamp: "Formatted at 2026-08-06 16:17:24 on dist-test-slave-t3q3"
server_key: "681f441dac20ac3064773a4ffd985a5e"
server_key_iv: "60aed10ab7f096d4fb4b335b0aeaf023"
server_key_version: "encryptionkey@0"
I20260806 16:17:24.907256 20510 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 16:17:24.910554 20531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:24.912719 20510 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 16:17:24.913090 20510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "d99db8b02533404fb487a2355255ec01"
format_stamp: "Formatted at 2026-08-06 16:17:24 on dist-test-slave-t3q3"
server_key: "681f441dac20ac3064773a4ffd985a5e"
server_key_iv: "60aed10ab7f096d4fb4b335b0aeaf023"
server_key_version: "encryptionkey@0"
I20260806 16:17:24.913431 20510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:24.930962 20510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:24.932013 20510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:24.932317 20510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:24.941212 20510 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:36909
I20260806 16:17:24.941282 20606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:36909 every 8 connection(s)
I20260806 16:17:24.942548 20510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:17:24.946447 20607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:24.948776 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 20510
I20260806 16:17:24.948946 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:17:24.949334 12246 external_mini_cluster.cc:1428] Setting key 42356e37860a861a4e5d1065d7b27074
I20260806 16:17:24.955302 20607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01: Bootstrap starting.
I20260806 16:17:24.958364 20607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:24.959292 20607 log.cc:824] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:24.960753 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:24.951176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39680 (local address 127.11.245.190:36909)
0806 16:17:24.952161 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:24.952183 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:24.952234 (+ 51us) server_negotiation.cc:407] Connection header received
0806 16:17:24.953048 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:24.953072 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:24.953601 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:24.954042 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:24.954597 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:24.955753 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:24.956766 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:24.957285 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:24.957486 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:24.958525 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:24.958533 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:24.958555 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:17:24.959048 (+ 493us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:24.959118 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:17:24.959140 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:24.959257 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:24.959435 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 16:17:24.959758 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":170,"threads_started":1}
I20260806 16:17:24.962217 20607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01: No bootstrap required, opened a new log
I20260806 16:17:24.964679 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:17:24.967079 20607 raft_consensus.cc:348] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99db8b02533404fb487a2355255ec01" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36909 } }
I20260806 16:17:24.967392 20607 raft_consensus.cc:374] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:24.967485 20607 raft_consensus.cc:729] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d99db8b02533404fb487a2355255ec01, State: Initialized, Role: FOLLOWER
I20260806 16:17:24.968014 20607 consensus_queue.cc:260] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99db8b02533404fb487a2355255ec01" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36909 } }
I20260806 16:17:24.968264 20607 raft_consensus.cc:388] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:17:24.968351 20607 raft_consensus.cc:482] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:17:24.968489 20607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:24.969655 20607 raft_consensus.cc:504] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99db8b02533404fb487a2355255ec01" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36909 } }
I20260806 16:17:24.970077 20607 leader_election.cc:302] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d99db8b02533404fb487a2355255ec01; no voters:
I20260806 16:17:24.970471 20607 leader_election.cc:288] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:17:24.970782 20617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:24.971118 20617 raft_consensus.cc:686] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [term 1 LEADER]: Becoming Leader. State: Replica: d99db8b02533404fb487a2355255ec01, State: Running, Role: LEADER
I20260806 16:17:24.971573 20617 consensus_queue.cc:237] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99db8b02533404fb487a2355255ec01" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36909 } }
I20260806 16:17:24.972285 20607 sys_catalog.cc:564] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:24.973965 20619 sys_catalog.cc:455] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d99db8b02533404fb487a2355255ec01. Latest consensus state: current_term: 1 leader_uuid: "d99db8b02533404fb487a2355255ec01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99db8b02533404fb487a2355255ec01" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36909 } } }
I20260806 16:17:24.974133 20619 sys_catalog.cc:458] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:24.974545 20618 sys_catalog.cc:455] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d99db8b02533404fb487a2355255ec01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99db8b02533404fb487a2355255ec01" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36909 } } }
I20260806 16:17:24.974694 20618 sys_catalog.cc:458] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01 [sys.catalog]: This master's current role is: LEADER
W20260806 16:17:24.979010 20634 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:17:24.979107 20634 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:17:24.979218 20629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:24.980152 20629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:24.987077 20629 catalog_manager.cc:1347] Generated new cluster ID: 1d92839ce12e4b2e8c0f2a904a208db0
I20260806 16:17:24.987185 20629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:25.016835 20629 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:17:25.018227 20629 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:25.027009 20629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d99db8b02533404fb487a2355255ec01: Generated new TSK 0
I20260806 16:17:25.027971 20629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:17:25.101001 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:25.037947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39688 (local address 127.11.245.190:36909)
0806 16:17:25.038124 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:25.038131 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:25.038306 (+ 175us) server_negotiation.cc:407] Connection header received
0806 16:17:25.038501 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:25.038505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:25.038579 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:25.038729 (+ 150us) server_negotiation.cc:226] Negotiated authn=JWT
0806 16:17:25.039090 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:25.040640 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:25.041630 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:25.041994 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:25.042107 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:25.042711 (+ 604us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 16:17:25.100263 (+ 57552us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 16:17:25.100773 (+ 510us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39688: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 16:17:25.101138 20614 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39688: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 16:17:25.101095 20651 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:25.037788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:36909 (local address 127.0.0.1:39688)
0806 16:17:25.038193 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:25.038208 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:25.038333 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:25.038767 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:25.038793 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:25.038803 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0806 16:17:25.038943 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:25.038953 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:25.040817 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:25.040820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:25.041495 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:25.041502 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:25.042160 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:25.042164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:25.042249 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:25.042452 (+ 203us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 16:17:25.100654 (+ 58202us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 16:17:25.100903 (+ 249us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.245.190:36909: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":142,"threads_started":1}
I20260806 16:17:25.102097 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 20510
2026-08-06T16:17:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (441 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (164 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (160 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:17:25 dist-test-slave-t3q3 krb5kdc[20669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:17:25 dist-test-slave-t3q3 krb5kdc[20669](info): set up 2 sockets
Aug 06 16:17:25 dist-test-slave-t3q3 krb5kdc[20669](info): commencing operation
krb5kdc: starting...
W20260806 16:17:26.745775 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:17:26 dist-test-slave-t3q3 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:17:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:17:26.900631 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:42561
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:46475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:42561
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:27.047587 20687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:27.048214 20687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:27.048379 20687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:27.055943 20687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:27.056137 20687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:27.056187 20687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:27.056241 20687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:27.056295 20687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:17:27.070297 20687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:46475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:42561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:42561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:27.072724 20687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:27.074700 20687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:27.089600 20698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:27.089600 20695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:27.089600 20694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:27.090407 20687 server_base.cc:1034] running on GCE node
I20260806 16:17:27.091308 20687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:27.094293 20687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:27.095738 20687 hybrid_clock.cc:648] HybridClock initialized: now 1786033047095669 us; error 58 us; skew 500 ppm
Aug 06 16:17:27 dist-test-slave-t3q3 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033047, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:27.101140 20687 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:17:27.102618 20687 webserver.cc:460] Webserver started at http://127.11.245.190:42113/ using document root <none> and password file <none>
I20260806 16:17:27.103459 20687 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:27.103569 20687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:27.103875 20687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:27.106858 20687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "f2a539f65d60486cb6a0d29e59e693b2"
format_stamp: "Formatted at 2026-08-06 16:17:27 on dist-test-slave-t3q3"
server_key: "ba79c231850f91983fdd46d08035c9fb"
server_key_iv: "3fff25a6df90dc23f6482394019adad7"
server_key_version: "encryptionkey@0"
I20260806 16:17:27.107934 20687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "f2a539f65d60486cb6a0d29e59e693b2"
format_stamp: "Formatted at 2026-08-06 16:17:27 on dist-test-slave-t3q3"
server_key: "ba79c231850f91983fdd46d08035c9fb"
server_key_iv: "3fff25a6df90dc23f6482394019adad7"
server_key_version: "encryptionkey@0"
I20260806 16:17:27.113579 20687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:17:27.117026 20709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:27.119212 20687 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:17:27.119390 20687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "f2a539f65d60486cb6a0d29e59e693b2"
format_stamp: "Formatted at 2026-08-06 16:17:27 on dist-test-slave-t3q3"
server_key: "ba79c231850f91983fdd46d08035c9fb"
server_key_iv: "3fff25a6df90dc23f6482394019adad7"
server_key_version: "encryptionkey@0"
I20260806 16:17:27.119583 20687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:27.138804 20687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:27.143441 20687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:27.143743 20687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:27.154150 20687 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:42561
I20260806 16:17:27.154161 20783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:42561 every 8 connection(s)
I20260806 16:17:27.155534 20687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:17:27.159686 20785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:27.162700 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 20687
I20260806 16:17:27.162894 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:17:27.163451 12246 external_mini_cluster.cc:1428] Setting key 9053e81baf25bbb215f76cfaaa1fe3d1
I20260806 16:17:27.168013 20785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2: Bootstrap starting.
I20260806 16:17:27.171628 20785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:27.172889 20785 log.cc:824] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:17:27 dist-test-slave-t3q3 krb5kdc[20669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:27.175902 20785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2: No bootstrap required, opened a new log
I20260806 16:17:27.180652 20785 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2a539f65d60486cb6a0d29e59e693b2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 42561 } }
I20260806 16:17:27.181008 20785 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:27.181098 20785 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2a539f65d60486cb6a0d29e59e693b2, State: Initialized, Role: FOLLOWER
I20260806 16:17:27.181684 20785 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2a539f65d60486cb6a0d29e59e693b2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 42561 } }
I20260806 16:17:27.181829 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:27.165518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46294 (local address 127.11.245.190:42561)
0806 16:17:27.166132 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:27.166152 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:27.166203 (+ 51us) server_negotiation.cc:407] Connection header received
0806 16:17:27.166994 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:27.167016 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:27.167458 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:27.167833 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:27.169032 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:27.170183 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:27.171446 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:27.172205 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:27.172382 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:27.175709 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:27.175716 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:27.175737 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:27.175771 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:27.177864 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:27.178482 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:27.178489 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:27.178495 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:27.178642 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:27.178986 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:27.178991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:27.178993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:27.179558 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:27.179847 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:27.180234 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 16:17:27.180651 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":236,"threads_started":1}
I20260806 16:17:27.181874 20785 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:17:27.181970 20785 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:17:27.182097 20785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:27.183652 20785 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2a539f65d60486cb6a0d29e59e693b2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 42561 } }
I20260806 16:17:27.184175 20785 leader_election.cc:302] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2a539f65d60486cb6a0d29e59e693b2; no voters:
I20260806 16:17:27.184659 20785 leader_election.cc:288] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:17:27.184754 20791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:27.184988 20791 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [term 1 LEADER]: Becoming Leader. State: Replica: f2a539f65d60486cb6a0d29e59e693b2, State: Running, Role: LEADER
I20260806 16:17:27.185384 20791 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2a539f65d60486cb6a0d29e59e693b2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 42561 } }
I20260806 16:17:27.186374 20785 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:27.187789 20794 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2a539f65d60486cb6a0d29e59e693b2. Latest consensus state: current_term: 1 leader_uuid: "f2a539f65d60486cb6a0d29e59e693b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2a539f65d60486cb6a0d29e59e693b2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 42561 } } }
I20260806 16:17:27.187979 20794 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:27.188429 20792 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2a539f65d60486cb6a0d29e59e693b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2a539f65d60486cb6a0d29e59e693b2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 42561 } } }
I20260806 16:17:27.188544 20792 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2 [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:27.188887 20805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:27.193859 20805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:27.203497 20805 catalog_manager.cc:1347] Generated new cluster ID: 0cb2db5b38864ab38bf66aa36a0461c9
I20260806 16:17:27.203637 20805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:27.224546 20805 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:17:27.226296 20805 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:27.237591 20805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2a539f65d60486cb6a0d29e59e693b2: Generated new TSK 0
I20260806 16:17:27.239042 20805 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:17:27.281579 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:42561
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:46475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:27.438949 20823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:27.439395 20823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:27.439489 20823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:27.443989 20823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:27.444160 20823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:27.444268 20823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:17:27.450897 20823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:46475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:42561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:27.452629 20823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:27.454111 20823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:27.466732 20835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:27.466763 20829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:27.466763 20832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:27.467099 20823 server_base.cc:1034] running on GCE node
I20260806 16:17:27.467630 20823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:27.468567 20823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:27.469801 20823 hybrid_clock.cc:648] HybridClock initialized: now 1786033047469754 us; error 70 us; skew 500 ppm
Aug 06 16:17:27 dist-test-slave-t3q3 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033047, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:27.474499 20823 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:17:27.475908 20823 webserver.cc:460] Webserver started at http://127.11.245.129:46533/ using document root <none> and password file <none>
I20260806 16:17:27.476799 20823 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:27.476912 20823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:27.477228 20823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:27.480252 20823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "9569fff265fe4f698f15f1ddac0c011d"
format_stamp: "Formatted at 2026-08-06 16:17:27 on dist-test-slave-t3q3"
server_key: "3bb5a2829982c551e921dce0ff4b8fc1"
server_key_iv: "d6338b0c3fbf966cbffaf01bdd9d97bf"
server_key_version: "encryptionkey@0"
I20260806 16:17:27.481112 20823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "9569fff265fe4f698f15f1ddac0c011d"
format_stamp: "Formatted at 2026-08-06 16:17:27 on dist-test-slave-t3q3"
server_key: "3bb5a2829982c551e921dce0ff4b8fc1"
server_key_iv: "d6338b0c3fbf966cbffaf01bdd9d97bf"
server_key_version: "encryptionkey@0"
I20260806 16:17:27.486095 20823 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:17:27.489727 20842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:27.491340 20823 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 16:17:27.491530 20823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "9569fff265fe4f698f15f1ddac0c011d"
format_stamp: "Formatted at 2026-08-06 16:17:27 on dist-test-slave-t3q3"
server_key: "3bb5a2829982c551e921dce0ff4b8fc1"
server_key_iv: "d6338b0c3fbf966cbffaf01bdd9d97bf"
server_key_version: "encryptionkey@0"
I20260806 16:17:27.491689 20823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:27.515900 20823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:27.520488 20823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:27.520784 20823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:27.521858 20823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:27.523094 20823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:27.523183 20823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:27.523270 20823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:27.523347 20823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:27.535218 20823 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:42149
I20260806 16:17:27.535245 20991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:42149 every 8 connection(s)
I20260806 16:17:27.536463 20823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:17:27.544803 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 20823
I20260806 16:17:27.544983 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:17:27.545377 12246 external_mini_cluster.cc:1428] Setting key 119f88a8b3a8ef7bc30bf6cad561a5eb
Aug 06 16:17:27 dist-test-slave-t3q3 krb5kdc[20669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033047, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:27.553869 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:27.538932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:56859 (local address 127.11.245.190:42561)
0806 16:17:27.539115 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:27.539119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:27.540180 (+ 1061us) server_negotiation.cc:407] Connection header received
0806 16:17:27.541848 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:27.541852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:27.541928 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:27.542048 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:27.543547 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:27.544247 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:27.545698 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:27.545955 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:27.546074 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:27.549573 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:27.549577 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:27.549580 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:27.549608 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:27.551239 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:27.551871 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:27.551875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:27.551877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:27.551974 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:27.552416 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:27.552421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:27.552423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:27.552661 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:27.552777 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:27.553487 (+ 710us) server_negotiation.cc:299] Negotiation successful
0806 16:17:27.553660 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 16:17:27.555063 20997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:27.539390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:42561 (local address 127.11.245.129:56859)
0806 16:17:27.539970 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:27.540021 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:27.541587 (+ 1566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:27.542270 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:27.542278 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:27.542727 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:27.543375 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:27.543391 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:27.544394 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:27.544398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:27.545561 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:27.545574 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:27.546139 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:27.546143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:27.546384 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:27.547253 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:27.547282 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:27.549362 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:27.551389 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:27.551397 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:27.551421 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:27.551741 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:27.552108 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:27.552114 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:27.552119 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:27.552306 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:27.552795 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:27.552804 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:27.553178 (+ 374us) client_negotiation.cc:769] Sending connection context
0806 16:17:27.553420 (+ 242us) client_negotiation.cc:240] Negotiation successful
0806 16:17:27.553885 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":143,"threads_started":1}
I20260806 16:17:27.557077 20992 heartbeater.cc:347] Connected to a master server at 127.11.245.190:42561
I20260806 16:17:27.557471 20992 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:27.558261 20992 heartbeater.cc:511] Master 127.11.245.190:42561 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:17:27.560738 20725 ts_manager.cc:194] Registered new tserver with Master: 9569fff265fe4f698f15f1ddac0c011d (127.11.245.129:42149)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
I20260806 16:17:27.562358 20725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:56859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:17:27.582103 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:42561
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:46475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:27.730587 21000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:27.731004 21000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:27.731089 21000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:27.735499 21000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:27.735586 21000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:27.735654 21000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:17:27.741909 21000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:46475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:42561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:27.743707 21000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:27.745281 21000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:27.758586 21008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:27.758587 21005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:27.758649 21006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:27.758809 21000 server_base.cc:1034] running on GCE node
I20260806 16:17:27.759536 21000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:27.760474 21000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:27.761695 21000 hybrid_clock.cc:648] HybridClock initialized: now 1786033047761675 us; error 35 us; skew 500 ppm
Aug 06 16:17:27 dist-test-slave-t3q3 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033047, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:27.765246 21000 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:17:27.766505 21000 webserver.cc:460] Webserver started at http://127.11.245.130:33113/ using document root <none> and password file <none>
I20260806 16:17:27.767284 21000 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:27.767390 21000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:27.767686 21000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:27.770390 21000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "b432630b7bcf47f9a048ee7a8a9bf819"
format_stamp: "Formatted at 2026-08-06 16:17:27 on dist-test-slave-t3q3"
server_key: "5d95edb43347be4287782d5d7f23a779"
server_key_iv: "e0b7d4c8083250698de500b10eb6ddc7"
server_key_version: "encryptionkey@0"
I20260806 16:17:27.771107 21000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "b432630b7bcf47f9a048ee7a8a9bf819"
format_stamp: "Formatted at 2026-08-06 16:17:27 on dist-test-slave-t3q3"
server_key: "5d95edb43347be4287782d5d7f23a779"
server_key_iv: "e0b7d4c8083250698de500b10eb6ddc7"
server_key_version: "encryptionkey@0"
I20260806 16:17:27.776165 21000 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:17:27.779520 21019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:27.781481 21000 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:17:27.781698 21000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "b432630b7bcf47f9a048ee7a8a9bf819"
format_stamp: "Formatted at 2026-08-06 16:17:27 on dist-test-slave-t3q3"
server_key: "5d95edb43347be4287782d5d7f23a779"
server_key_iv: "e0b7d4c8083250698de500b10eb6ddc7"
server_key_version: "encryptionkey@0"
I20260806 16:17:27.781859 21000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:27.806998 21000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:27.811405 21000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:27.811702 21000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:27.812608 21000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:27.813812 21000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:27.813869 21000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:27.813923 21000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:27.813942 21000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:27.826088 21000 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:35753
I20260806 16:17:27.826205 21178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:35753 every 8 connection(s)
I20260806 16:17:27.827343 21000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:17:27.833446 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 21000
I20260806 16:17:27.833622 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:17:27.834189 12246 external_mini_cluster.cc:1428] Setting key 77bfc79e196d9468ad52077755098d53
Aug 06 16:17:27 dist-test-slave-t3q3 krb5kdc[20669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033047, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:27.845278 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:27.829966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:43935 (local address 127.11.245.190:42561)
0806 16:17:27.830167 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:27.830172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:27.831328 (+ 1156us) server_negotiation.cc:407] Connection header received
0806 16:17:27.833081 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:27.833086 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:27.833167 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:27.833285 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:27.835125 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:27.835728 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:27.837238 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:27.837493 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:27.837616 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:27.840712 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:27.840715 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:27.840718 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:27.840748 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:27.842464 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:27.843139 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:27.843143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:27.843144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:27.843233 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:27.843642 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:27.843645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:27.843647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:27.843904 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:27.844028 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:27.844783 (+ 755us) server_negotiation.cc:299] Negotiation successful
0806 16:17:27.845069 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 16:17:27.846473 21185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:27.830451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:42561 (local address 127.11.245.130:43935)
0806 16:17:27.831121 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:27.831176 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:27.832790 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:27.833560 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:27.833571 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:27.834100 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:27.834915 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:27.834945 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:27.835886 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:27.835890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:27.837090 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:27.837102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:27.837700 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:27.837704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:27.837922 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:27.838694 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:27.838715 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:27.840494 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:27.842625 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:27.842633 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:27.842659 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:27.843001 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:27.843334 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:27.843337 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:27.843339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:27.843530 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:27.844028 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:27.844038 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:27.844466 (+ 428us) client_negotiation.cc:769] Sending connection context
0806 16:17:27.844720 (+ 254us) client_negotiation.cc:240] Negotiation successful
0806 16:17:27.845296 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":161,"threads_started":1}
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
I20260806 16:17:27.848271 21180 heartbeater.cc:347] Connected to a master server at 127.11.245.190:42561
I20260806 16:17:27.848630 21180 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:27.849349 21180 heartbeater.cc:511] Master 127.11.245.190:42561 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
I20260806 16:17:27.850900 20725 ts_manager.cc:194] Registered new tserver with Master: b432630b7bcf47f9a048ee7a8a9bf819 (127.11.245.130:35753)
I20260806 16:17:27.851871 20725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:43935
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:17:27.871853 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:42561
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:46475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:28.021059 21188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:28.021453 21188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:28.021509 21188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:28.026043 21188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:28.026154 21188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:28.026227 21188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:17:28.032444 21188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:46475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:42561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:28.034005 21188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:28.035409 21188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:28.048384 21196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:28.048521 21200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:28.048384 21197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:28.048998 21188 server_base.cc:1034] running on GCE node
I20260806 16:17:28.049621 21188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:28.050632 21188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:28.051981 21188 hybrid_clock.cc:648] HybridClock initialized: now 1786033048051951 us; error 42 us; skew 500 ppm
Aug 06 16:17:28 dist-test-slave-t3q3 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033048, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:28.055989 21188 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:17:28.057319 21188 webserver.cc:460] Webserver started at http://127.11.245.131:38379/ using document root <none> and password file <none>
I20260806 16:17:28.058126 21188 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:28.058235 21188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:28.058542 21188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:28.061354 21188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "6cd71b78d7a74151991eb05a00cf4e0b"
format_stamp: "Formatted at 2026-08-06 16:17:28 on dist-test-slave-t3q3"
server_key: "09806365c76b127d960629974a222c44"
server_key_iv: "7df53008926ee87cb3a2028b58442a17"
server_key_version: "encryptionkey@0"
I20260806 16:17:28.062089 21188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "6cd71b78d7a74151991eb05a00cf4e0b"
format_stamp: "Formatted at 2026-08-06 16:17:28 on dist-test-slave-t3q3"
server_key: "09806365c76b127d960629974a222c44"
server_key_iv: "7df53008926ee87cb3a2028b58442a17"
server_key_version: "encryptionkey@0"
I20260806 16:17:28.067076 21188 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:17:28.070272 21207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:28.071863 21188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:28.072041 21188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "6cd71b78d7a74151991eb05a00cf4e0b"
format_stamp: "Formatted at 2026-08-06 16:17:28 on dist-test-slave-t3q3"
server_key: "09806365c76b127d960629974a222c44"
server_key_iv: "7df53008926ee87cb3a2028b58442a17"
server_key_version: "encryptionkey@0"
I20260806 16:17:28.072253 21188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:28.084605 21188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:28.089300 21188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:28.089576 21188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:28.090381 21188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:28.091475 21188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:28.091552 21188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:28.091637 21188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:28.091692 21188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:28.105607 21188 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:41909
I20260806 16:17:28.105633 21387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:41909 every 8 connection(s)
I20260806 16:17:28.106940 21188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:17:28.113741 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 21188
I20260806 16:17:28.113922 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:17:28.114301 12246 external_mini_cluster.cc:1428] Setting key 23aa494fed413857bc2c03bd6008066e
Aug 06 16:17:28 dist-test-slave-t3q3 krb5kdc[20669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033048, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:28.125211 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:28.109516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:53849 (local address 127.11.245.190:42561)
0806 16:17:28.109695 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:28.109699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:28.110805 (+ 1106us) server_negotiation.cc:407] Connection header received
0806 16:17:28.112594 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:28.112598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:28.112660 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:28.112736 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:28.114427 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:28.115095 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:28.117046 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:28.117446 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:28.117573 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:28.120847 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:28.120851 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:28.120853 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:28.120884 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:28.122610 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:28.123233 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:28.123236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:28.123237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:28.123363 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:28.123768 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:28.123771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:28.123772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:28.124025 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:28.124224 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:28.124922 (+ 698us) server_negotiation.cc:299] Negotiation successful
0806 16:17:28.125052 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 16:17:28.126313 21391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:28.109952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:42561 (local address 127.11.245.131:53849)
0806 16:17:28.110596 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:28.110651 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:28.112278 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:28.112962 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:28.112969 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:28.113468 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:28.114233 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:28.114254 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:28.115270 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:28.115274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:28.116880 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:28.116893 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:28.117629 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:28.117632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:28.117816 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:28.118684 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:28.118703 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:28.120628 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:28.122747 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:28.122752 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:28.122772 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:28.123098 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:28.123499 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:28.123503 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:28.123504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:28.123677 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:28.124228 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:28.124234 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:28.124550 (+ 316us) client_negotiation.cc:769] Sending connection context
0806 16:17:28.124747 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 16:17:28.125108 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":154,"threads_started":1}
I20260806 16:17:28.128129 21388 heartbeater.cc:347] Connected to a master server at 127.11.245.190:42561
I20260806 16:17:28.128554 21388 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:28.129437 21388 heartbeater.cc:511] Master 127.11.245.190:42561 requested a full tablet report, sending...
I20260806 16:17:28.130836 20725 ts_manager.cc:194] Registered new tserver with Master: 6cd71b78d7a74151991eb05a00cf4e0b (127.11.245.131:41909)
I20260806 16:17:28.131531 20725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:53849
I20260806 16:17:28.143267 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/test-admin.keytab.
Aug 06 16:17:28 dist-test-slave-t3q3 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:28.165096 12246 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 16:17:28.169327 21398 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:28.167281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42298 (local address 127.11.245.129:42149)
0806 16:17:28.167730 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:28.167742 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:28.167779 (+ 37us) server_negotiation.cc:407] Connection header received
0806 16:17:28.167856 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:28.167862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:28.168202 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:28.168364 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:28.169086 (+ 722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42298: BlockingRecv error: recv got EOF from 127.0.0.1:42298 (error 108)
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":217,"threads_started":1}
W20260806 16:17:28.169901 12246 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/krb5cc)
Aug 06 16:17:28 dist-test-slave-t3q3 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:28.171321 12246 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 16:17:28 dist-test-slave-t3q3 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:28.172704 12246 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 16:17:28 dist-test-slave-t3q3 krb5kdc[20669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.129@KRBTEST.COM
I20260806 16:17:28.186657 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:28.174160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42314 (local address 127.11.245.129:42149)
0806 16:17:28.174339 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:28.174347 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:28.174458 (+ 111us) server_negotiation.cc:407] Connection header received
0806 16:17:28.174655 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:28.174660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:28.174731 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:28.174830 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:28.175520 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:28.176896 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:28.177892 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:28.178375 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:28.178509 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:28.181549 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:28.181562 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:28.181585 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:28.181623 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:28.183767 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:28.184829 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:28.184838 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:28.184839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:28.184946 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:28.185302 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:28.185306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:28.185310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:28.185681 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:28.186015 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:28.186193 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 16:17:28.186405 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 16:17:28.193429 21398 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:28.191896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42318 (local address 127.11.245.129:42149)
0806 16:17:28.192138 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:28.192145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:28.192311 (+ 166us) server_negotiation.cc:407] Connection header received
0806 16:17:28.192519 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:28.192523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:28.192586 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:28.192683 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:28.193257 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42318: BlockingRecv error: recv got EOF from 127.0.0.1:42318 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260806 16:17:28.193579 12246 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/krb5cc' of type FILE
Aug 06 16:17:28 dist-test-slave-t3q3 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:28.194986 12246 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 16:17:28 dist-test-slave-t3q3 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:28.196277 12246 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 16:17:28 dist-test-slave-t3q3 krb5kdc[20669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.129@KRBTEST.COM
I20260806 16:17:28.207489 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:28.197604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42320 (local address 127.11.245.129:42149)
0806 16:17:28.197764 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:28.197772 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:28.197862 (+ 90us) server_negotiation.cc:407] Connection header received
0806 16:17:28.198055 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:28.198058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:28.198123 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:28.198223 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:28.198984 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:28.199561 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:28.200557 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:28.200812 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:28.200947 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:28.203807 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:28.203811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:28.203813 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:28.203868 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:28.205525 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:28.206069 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:28.206080 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:28.206082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:28.206170 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:28.206475 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:28.206479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:28.206481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:28.206701 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:28.206885 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:28.207139 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 16:17:28.207286 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 16:17:28.211555 21398 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:28.210275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42328 (local address 127.11.245.129:42149)
0806 16:17:28.210504 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:28.210507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:28.210542 (+ 35us) server_negotiation.cc:407] Connection header received
0806 16:17:28.210694 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:28.210697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:28.210741 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:28.210809 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:28.211388 (+ 579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42328: BlockingRecv error: recv got EOF from 127.0.0.1:42328 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 16:17:28.211650 12246 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786033011181791-12246-0/krb5kdc/krb5cc' of type FILE
Aug 06 16:17:28 dist-test-slave-t3q3 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:28.213064 12246 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 16:17:28 dist-test-slave-t3q3 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:28.214164 12246 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 16:17:28 dist-test-slave-t3q3 krb5kdc[20669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.129@KRBTEST.COM
I20260806 16:17:28.226156 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:28.215346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42344 (local address 127.11.245.129:42149)
0806 16:17:28.215542 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:28.215547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:28.215700 (+ 153us) server_negotiation.cc:407] Connection header received
0806 16:17:28.215766 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:28.215769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:28.215819 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:28.215904 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:28.216668 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:28.217242 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:28.218204 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:28.218473 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:28.218577 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:28.221757 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:28.221763 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:28.221765 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:28.221806 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:28.223519 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:28.224294 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:28.224298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:28.224300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:28.224494 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:28.224904 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:28.224907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:28.224909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:28.225160 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:28.225358 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:28.225734 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 16:17:28.225918 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 16:17:28.227950 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 20823
I20260806 16:17:28.250546 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 21000
I20260806 16:17:28.267426 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 21188
I20260806 16:17:28.285650 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 20687
2026-08-06T16:17:28Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2849 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:17:28 dist-test-slave-t3q3 krb5kdc[21415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:17:28 dist-test-slave-t3q3 krb5kdc[21415](info): set up 2 sockets
Aug 06 16:17:28 dist-test-slave-t3q3 krb5kdc[21415](info): commencing operation
krb5kdc: starting...
W20260806 16:17:29.601711 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:17:29 dist-test-slave-t3q3 krb5kdc[21415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:17:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:29Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.190.keytab.
Entry for principal oryx/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.190.keytab.
I20260806 16:17:29.768632 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:37595
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.190.keytab
--principal=oryx/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:37595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:29.910782 21436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:29.911157 21436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:29.911203 21436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:29.915809 21436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:29.915939 21436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:29.915962 21436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:29.915982 21436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:29.915998 21436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:17:29.922251 21436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:37595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.190.keytab
--principal=oryx/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:37595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:29.923708 21436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:29.925143 21436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:29.936368 21441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:29.936450 21436 server_base.cc:1034] running on GCE node
W20260806 16:17:29.936368 21442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:29.936365 21445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:29.937402 21436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:29.938936 21436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:29.940269 21436 hybrid_clock.cc:648] HybridClock initialized: now 1786033049940245 us; error 40 us; skew 500 ppm
Aug 06 16:17:29 dist-test-slave-t3q3 krb5kdc[21415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033049, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:29.943780 21436 init.cc:377] Logged in from keytab as oryx/127.11.245.190@KRBTEST.COM (short username oryx)
I20260806 16:17:29.945071 21436 webserver.cc:460] Webserver started at http://127.11.245.190:43655/ using document root <none> and password file <none>
I20260806 16:17:29.945822 21436 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:29.945896 21436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:29.946175 21436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:29.949040 21436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "4097a09218064c698b4d0b89cffba1ff"
format_stamp: "Formatted at 2026-08-06 16:17:29 on dist-test-slave-t3q3"
server_key: "f47a231a305c103e95bdfd5be2e48f08"
server_key_iv: "1b8dba738622b65d07f241a418ea0238"
server_key_version: "encryptionkey@0"
I20260806 16:17:29.949779 21436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "4097a09218064c698b4d0b89cffba1ff"
format_stamp: "Formatted at 2026-08-06 16:17:29 on dist-test-slave-t3q3"
server_key: "f47a231a305c103e95bdfd5be2e48f08"
server_key_iv: "1b8dba738622b65d07f241a418ea0238"
server_key_version: "encryptionkey@0"
I20260806 16:17:29.955155 21436 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:17:29.958467 21455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:29.960165 21436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:17:29.960335 21436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "4097a09218064c698b4d0b89cffba1ff"
format_stamp: "Formatted at 2026-08-06 16:17:29 on dist-test-slave-t3q3"
server_key: "f47a231a305c103e95bdfd5be2e48f08"
server_key_iv: "1b8dba738622b65d07f241a418ea0238"
server_key_version: "encryptionkey@0"
I20260806 16:17:29.960482 21436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:29.975973 21436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:29.980147 21436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:29.980404 21436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:29.989243 21436 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:37595
I20260806 16:17:29.989589 21530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:37595 every 8 connection(s)
I20260806 16:17:29.990543 21436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:17:29.994336 21532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:29.999151 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 21436
I20260806 16:17:29.999337 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:17:29.999748 12246 external_mini_cluster.cc:1428] Setting key de5009301a763a14bf97d771c8cea522
I20260806 16:17:30.002194 21532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff: Bootstrap starting.
I20260806 16:17:30.005607 21532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:30.006637 21532 log.cc:824] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:30.010080 21532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff: No bootstrap required, opened a new log
Aug 06 16:17:30 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM
I20260806 16:17:30.014933 21532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } }
I20260806 16:17:30.015290 21532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:30.015386 21532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4097a09218064c698b4d0b89cffba1ff, State: Initialized, Role: FOLLOWER
I20260806 16:17:30.016043 21532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } }
I20260806 16:17:30.016317 21532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:17:30.016397 21532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:17:30.016541 21532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:30.017810 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:30.001669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45210 (local address 127.11.245.190:37595)
0806 16:17:30.002370 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:30.002403 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:30.002450 (+ 47us) server_negotiation.cc:407] Connection header received
0806 16:17:30.003275 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:30.003299 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:30.003739 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:30.004196 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:30.005180 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:30.006413 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:30.007547 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:30.008237 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:30.008385 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:30.011765 (+ 3380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:30.011772 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:30.011792 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:30.011827 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:30.013979 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:30.014537 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:30.014543 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:30.014549 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:30.014688 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:30.015023 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:30.015031 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:30.015033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:30.015655 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:30.015907 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:30.016258 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 16:17:30.016839 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":330,"threads_started":1}
I20260806 16:17:30.017807 21532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } }
I20260806 16:17:30.018410 21532 leader_election.cc:302] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4097a09218064c698b4d0b89cffba1ff; no voters:
I20260806 16:17:30.018867 21532 leader_election.cc:288] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:17:30.018997 21540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:30.019258 21540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [term 1 LEADER]: Becoming Leader. State: Replica: 4097a09218064c698b4d0b89cffba1ff, State: Running, Role: LEADER
I20260806 16:17:30.019639 21540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } }
I20260806 16:17:30.020574 21532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:30.021917 21541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4097a09218064c698b4d0b89cffba1ff. Latest consensus state: current_term: 1 leader_uuid: "4097a09218064c698b4d0b89cffba1ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } }
I20260806 16:17:30.021963 21542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4097a09218064c698b4d0b89cffba1ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } }
I20260806 16:17:30.022197 21541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:30.022229 21542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:30.023012 21552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:30.027072 21552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:30.034547 21552 catalog_manager.cc:1347] Generated new cluster ID: 36d1ca61ba534096b50a4019dd61d93b
I20260806 16:17:30.034673 21552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:30.048683 21552 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:17:30.050007 21552 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:30.061610 21552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff: Generated new TSK 0
I20260806 16:17:30.062654 21552 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.129.keytab.
Entry for principal oryx/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.129.keytab.
I20260806 16:17:30.107618 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:37595
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.129.keytab
--principal=oryx/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:30.282168 21568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:30.282670 21568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:30.282727 21568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:30.287401 21568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:30.287515 21568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:30.287586 21568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:17:30.294245 21568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.129.keytab
--principal=oryx/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:37595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:30.296015 21568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:30.297516 21568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:30.310554 21579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:30.310756 21568 server_base.cc:1034] running on GCE node
W20260806 16:17:30.310554 21581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:30.311012 21578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:30.311594 21568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:30.312572 21568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:30.313771 21568 hybrid_clock.cc:648] HybridClock initialized: now 1786033050313754 us; error 33 us; skew 500 ppm
Aug 06 16:17:30 dist-test-slave-t3q3 krb5kdc[21415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033050, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:30.317670 21568 init.cc:377] Logged in from keytab as oryx/127.11.245.129@KRBTEST.COM (short username oryx)
I20260806 16:17:30.318950 21568 webserver.cc:460] Webserver started at http://127.11.245.129:35749/ using document root <none> and password file <none>
I20260806 16:17:30.319809 21568 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:30.319977 21568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:30.320345 21568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:30.323493 21568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "f210e2981cc248e9a53de344a59a08ee"
format_stamp: "Formatted at 2026-08-06 16:17:30 on dist-test-slave-t3q3"
server_key: "6fbe05549491346155d5914caac3789b"
server_key_iv: "ae41b5124f958c4a4a0b5e3ab0934539"
server_key_version: "encryptionkey@0"
I20260806 16:17:30.324316 21568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "f210e2981cc248e9a53de344a59a08ee"
format_stamp: "Formatted at 2026-08-06 16:17:30 on dist-test-slave-t3q3"
server_key: "6fbe05549491346155d5914caac3789b"
server_key_iv: "ae41b5124f958c4a4a0b5e3ab0934539"
server_key_version: "encryptionkey@0"
I20260806 16:17:30.329499 21568 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:17:30.332898 21590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:30.334713 21568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:17:30.334887 21568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "f210e2981cc248e9a53de344a59a08ee"
format_stamp: "Formatted at 2026-08-06 16:17:30 on dist-test-slave-t3q3"
server_key: "6fbe05549491346155d5914caac3789b"
server_key_iv: "ae41b5124f958c4a4a0b5e3ab0934539"
server_key_version: "encryptionkey@0"
I20260806 16:17:30.335094 21568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:30.347797 21568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:30.352218 21568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:30.352478 21568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:30.353317 21568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:30.354442 21568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:30.354494 21568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:30.354540 21568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:30.354593 21568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:30.366712 21568 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:39661
I20260806 16:17:30.366736 21744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:39661 every 8 connection(s)
I20260806 16:17:30.367966 21568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:17:30.369951 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 21568
I20260806 16:17:30.370137 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:17:30.370589 12246 external_mini_cluster.cc:1428] Setting key 45942f7ebebb1e4b7fffbb6680e952b1
Aug 06 16:17:30 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033050, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.11.245.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.245.130@KRBTEST.COM" created.
I20260806 16:17:30.390760 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:30.370856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:41143 (local address 127.11.245.190:37595)
0806 16:17:30.371075 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:30.371081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:30.372401 (+ 1320us) server_negotiation.cc:407] Connection header received
0806 16:17:30.374401 (+ 2000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:30.374405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:30.374475 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:30.374602 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:30.376862 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:30.377489 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:30.379722 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:30.380043 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:30.380241 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:30.385059 (+ 4818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:30.385096 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:30.385102 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:30.385168 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:30.387429 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:30.388333 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:30.388341 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:30.388343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:30.388476 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:30.388952 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:30.388955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:30.388956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:30.389280 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:30.389376 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:30.390224 (+ 848us) server_negotiation.cc:299] Negotiation successful
0806 16:17:30.390431 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 16:17:30.391765 21751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:30.371413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.129:41143)
0806 16:17:30.372204 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:30.372277 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:30.374118 (+ 1841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:30.374844 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:30.374853 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:30.375560 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:30.376612 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:30.376646 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:30.377663 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:30.377671 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:30.379512 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:30.379534 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:30.380341 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:30.380349 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:30.380656 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:30.381578 (+ 922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:30.381614 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:30.384622 (+ 3008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:30.387631 (+ 3009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:30.387641 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:30.387666 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:30.388103 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:30.388584 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:30.388587 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:30.388588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:30.388801 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:30.389408 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:30.389421 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:30.389881 (+ 460us) client_negotiation.cc:769] Sending connection context
0806 16:17:30.390106 (+ 225us) client_negotiation.cc:240] Negotiation successful
0806 16:17:30.390499 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":180,"threads_started":1}
I20260806 16:17:30.393769 21747 heartbeater.cc:347] Connected to a master server at 127.11.245.190:37595
I20260806 16:17:30.394141 21747 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:30.394873 21747 heartbeater.cc:511] Master 127.11.245.190:37595 requested a full tablet report, sending...
I20260806 16:17:30.397395 21482 ts_manager.cc:194] Registered new tserver with Master: f210e2981cc248e9a53de344a59a08ee (127.11.245.129:39661)
I20260806 16:17:30.399127 21482 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:41143
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.130.keytab.
Entry for principal oryx/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.130.keytab.
I20260806 16:17:30.412794 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:37595
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.130.keytab
--principal=oryx/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:30.568405 21758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:30.568869 21758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:30.568991 21758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:30.573719 21758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:30.573853 21758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:30.573961 21758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:17:30.580451 21758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.130.keytab
--principal=oryx/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:37595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:30.582011 21758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:30.583664 21758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:30.596532 21765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:30.596498 21764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:30.596576 21758 server_base.cc:1034] running on GCE node
W20260806 16:17:30.596482 21768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:30.597357 21758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:30.598264 21758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:30.599462 21758 hybrid_clock.cc:648] HybridClock initialized: now 1786033050599425 us; error 53 us; skew 500 ppm
Aug 06 16:17:30 dist-test-slave-t3q3 krb5kdc[21415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033050, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:30.603437 21758 init.cc:377] Logged in from keytab as oryx/127.11.245.130@KRBTEST.COM (short username oryx)
I20260806 16:17:30.604847 21758 webserver.cc:460] Webserver started at http://127.11.245.130:36225/ using document root <none> and password file <none>
I20260806 16:17:30.605628 21758 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:30.605728 21758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:30.606134 21758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:30.609486 21758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "48485d69c35d4df58077716377f9f008"
format_stamp: "Formatted at 2026-08-06 16:17:30 on dist-test-slave-t3q3"
server_key: "b62f4d6aeaf0b09b9915f767eaf0e1a0"
server_key_iv: "f0297311b78f25905fdc9d6c1a6013a5"
server_key_version: "encryptionkey@0"
I20260806 16:17:30.610301 21758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "48485d69c35d4df58077716377f9f008"
format_stamp: "Formatted at 2026-08-06 16:17:30 on dist-test-slave-t3q3"
server_key: "b62f4d6aeaf0b09b9915f767eaf0e1a0"
server_key_iv: "f0297311b78f25905fdc9d6c1a6013a5"
server_key_version: "encryptionkey@0"
I20260806 16:17:30.616328 21758 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260806 16:17:30.620651 21775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:30.623505 21758 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.002s
I20260806 16:17:30.623698 21758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "48485d69c35d4df58077716377f9f008"
format_stamp: "Formatted at 2026-08-06 16:17:30 on dist-test-slave-t3q3"
server_key: "b62f4d6aeaf0b09b9915f767eaf0e1a0"
server_key_iv: "f0297311b78f25905fdc9d6c1a6013a5"
server_key_version: "encryptionkey@0"
I20260806 16:17:30.623871 21758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:30.666435 21758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:30.673161 21758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:30.673465 21758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:30.674499 21758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:30.675824 21758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:30.675932 21758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:30.676024 21758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:30.676095 21758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 16:17:30.692337 21758 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:46415
I20260806 16:17:30.692607 21930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:46415 every 8 connection(s)
I20260806 16:17:30.693717 21758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:17:30.695259 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 21758
I20260806 16:17:30.695426 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:17:30.695869 12246 external_mini_cluster.cc:1428] Setting key 9c056740c0da9ab1b33fdd4dc0dacb8a
Aug 06 16:17:30 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033050, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.11.245.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.245.131@KRBTEST.COM" created.
I20260806 16:17:30.716233 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:30.697302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:49649 (local address 127.11.245.190:37595)
0806 16:17:30.697513 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:30.697520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:30.698438 (+ 918us) server_negotiation.cc:407] Connection header received
0806 16:17:30.700348 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:30.700354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:30.700439 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:30.700554 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:30.702394 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:30.703330 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:30.704915 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:30.705376 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:30.705549 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:30.709585 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:30.709592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:30.709598 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:30.709651 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:30.711636 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:30.712977 (+ 1341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:30.712984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:30.712986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:30.713127 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:30.713691 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:30.713695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:30.713696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:30.714190 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:30.714313 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:30.715578 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0806 16:17:30.715917 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 16:17:30.717787 21935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:30.697518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.130:49649)
0806 16:17:30.698226 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:30.698307 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:30.699958 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:30.700831 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:30.700841 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:30.701496 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:30.702190 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:30.702208 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:30.703531 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:30.703535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:30.704745 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:30.704760 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:30.705650 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:30.705654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:30.705861 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:30.706721 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:30.706774 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:30.709257 (+ 2483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:30.711871 (+ 2614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:30.711880 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:30.711906 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:30.712546 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:30.713280 (+ 734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:30.713284 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:30.713286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:30.713527 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:30.714461 (+ 934us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:30.714472 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:30.715161 (+ 689us) client_negotiation.cc:769] Sending connection context
0806 16:17:30.715490 (+ 329us) client_negotiation.cc:240] Negotiation successful
0806 16:17:30.716008 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":203,"threads_started":1}
I20260806 16:17:30.719571 21933 heartbeater.cc:347] Connected to a master server at 127.11.245.190:37595
I20260806 16:17:30.720041 21933 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:30.720929 21933 heartbeater.cc:511] Master 127.11.245.190:37595 requested a full tablet report, sending...
I20260806 16:17:30.722680 21482 ts_manager.cc:194] Registered new tserver with Master: 48485d69c35d4df58077716377f9f008 (127.11.245.130:46415)
I20260806 16:17:30.723651 21482 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:49649
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.131.keytab.
Entry for principal oryx/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.131.keytab.
I20260806 16:17:30.740661 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:37595
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.131.keytab
--principal=oryx/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:30.896494 21941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:30.896855 21941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:30.896905 21941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:30.901258 21941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:30.901350 21941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:30.901419 21941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:17:30.908025 21941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.131.keytab
--principal=oryx/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:37595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:30.909657 21941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:30.911010 21941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:30.924767 21950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:30.924829 21947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:30.924829 21946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:30.925449 21941 server_base.cc:1034] running on GCE node
I20260806 16:17:30.926064 21941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:30.927039 21941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:30.928259 21941 hybrid_clock.cc:648] HybridClock initialized: now 1786033050928234 us; error 43 us; skew 500 ppm
Aug 06 16:17:30 dist-test-slave-t3q3 krb5kdc[21415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033050, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:30.932643 21941 init.cc:377] Logged in from keytab as oryx/127.11.245.131@KRBTEST.COM (short username oryx)
I20260806 16:17:30.933990 21941 webserver.cc:460] Webserver started at http://127.11.245.131:42023/ using document root <none> and password file <none>
I20260806 16:17:30.934819 21941 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:30.934926 21941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:30.935222 21941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:30.937996 21941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "974c509eec194b33b2b65b24a2ed72bb"
format_stamp: "Formatted at 2026-08-06 16:17:30 on dist-test-slave-t3q3"
server_key: "3e32af57b3fa02f59136e391b7f5c914"
server_key_iv: "20c55b84a54ff58f03a27633c3cd2281"
server_key_version: "encryptionkey@0"
I20260806 16:17:30.938737 21941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "974c509eec194b33b2b65b24a2ed72bb"
format_stamp: "Formatted at 2026-08-06 16:17:30 on dist-test-slave-t3q3"
server_key: "3e32af57b3fa02f59136e391b7f5c914"
server_key_iv: "20c55b84a54ff58f03a27633c3cd2281"
server_key_version: "encryptionkey@0"
I20260806 16:17:30.944547 21941 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 16:17:30.948150 21958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:30.950136 21941 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:17:30.950356 21941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "974c509eec194b33b2b65b24a2ed72bb"
format_stamp: "Formatted at 2026-08-06 16:17:30 on dist-test-slave-t3q3"
server_key: "3e32af57b3fa02f59136e391b7f5c914"
server_key_iv: "20c55b84a54ff58f03a27633c3cd2281"
server_key_version: "encryptionkey@0"
I20260806 16:17:30.950560 21941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:30.965744 21941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:30.970346 21941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:30.970670 21941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:30.971553 21941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:30.973043 21941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:30.973120 21941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:30.973201 21941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:30.973233 21941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:30.985845 21941 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:44719
I20260806 16:17:30.985867 22123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:44719 every 8 connection(s)
I20260806 16:17:30.987434 21941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:17:30.992245 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 21941
I20260806 16:17:30.992403 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:17:30.992801 12246 external_mini_cluster.cc:1428] Setting key 1418857d99d028dfbb1cc9bb9ddfe33e
Aug 06 16:17:30 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033050, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM
I20260806 16:17:31.009209 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:30.990288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:60475 (local address 127.11.245.190:37595)
0806 16:17:30.990483 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:30.990489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:30.991718 (+ 1229us) server_negotiation.cc:407] Connection header received
0806 16:17:30.993773 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:30.993778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:30.993846 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:30.993952 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:30.996183 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:30.996835 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:30.998793 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:30.999042 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:30.999144 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.003717 (+ 4573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.003723 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.003725 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.003759 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.005576 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.006409 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.006413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.006417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.006528 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.007317 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.007326 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.007328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.007594 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.007725 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.008784 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.008997 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 16:17:31.010550 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:30.990845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.131:60475)
0806 16:17:30.991499 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:30.991563 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:30.993494 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:30.994355 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:30.994366 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:30.994995 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:30.995909 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:30.995933 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:30.996985 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:30.996989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:30.998623 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:30.998636 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:30.999247 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:30.999251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:30.999492 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.000985 (+ 1493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:31.001012 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:31.003444 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:31.005795 (+ 2351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.005805 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.005825 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.006190 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.006893 (+ 703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.006896 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.006898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.007164 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.007728 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:31.007741 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:31.008320 (+ 579us) client_negotiation.cc:769] Sending connection context
0806 16:17:31.008746 (+ 426us) client_negotiation.cc:240] Negotiation successful
0806 16:17:31.009271 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":189,"threads_started":1}
I20260806 16:17:31.012212 22124 heartbeater.cc:347] Connected to a master server at 127.11.245.190:37595
I20260806 16:17:31.012580 22124 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:31.013370 22124 heartbeater.cc:511] Master 127.11.245.190:37595 requested a full tablet report, sending...
I20260806 16:17:31.014945 21482 ts_manager.cc:194] Registered new tserver with Master: 974c509eec194b33b2b65b24a2ed72bb (127.11.245.131:44719)
I20260806 16:17:31.015723 21482 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:60475
I20260806 16:17:31.019987 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.245.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.245.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.245.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.189.keytab.
Entry for principal oryx/127.11.245.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.189.keytab.
I20260806 16:17:31.060632 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.189:37321
--webserver_interface=127.11.245.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.189.keytab
--principal=oryx/127.11.245.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:37595,127.11.245.189:37321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:31.221786 22131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:31.222910 22131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:31.223083 22131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:31.228214 22131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:31.228315 22131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:31.228338 22131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:31.228356 22131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:31.228374 22131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:17:31.235055 22131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:37595,127.11.245.189:37321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.189.keytab
--principal=oryx/127.11.245.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.189:37321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.245.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
W20260806 16:17:31.235384 22131 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.245.190:37595,127.11.245.189:37321'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 16:17:31.236742 22131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:31.238133 22131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:31.249739 22143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:31.249739 22139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:31.249768 22140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:31.249928 22131 server_base.cc:1034] running on GCE node
I20260806 16:17:31.251017 22131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:31.252708 22131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:31.253921 22131 hybrid_clock.cc:648] HybridClock initialized: now 1786033051253880 us; error 60 us; skew 500 ppm
Aug 06 16:17:31 dist-test-slave-t3q3 krb5kdc[21415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033051, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:31.257706 22131 init.cc:377] Logged in from keytab as oryx/127.11.245.189@KRBTEST.COM (short username oryx)
I20260806 16:17:31.259048 22131 webserver.cc:460] Webserver started at http://127.11.245.189:44053/ using document root <none> and password file <none>
I20260806 16:17:31.259867 22131 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:31.259979 22131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:31.260344 22131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:31.263561 22131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/data/instance:
uuid: "fcb849e01bf14ac29f38d298b756367a"
format_stamp: "Formatted at 2026-08-06 16:17:31 on dist-test-slave-t3q3"
server_key: "3700d3f890b598c8c0f58e5ece24b0c6"
server_key_iv: "b6ba7836d8908cc6acd3c2b0686a2382"
server_key_version: "encryptionkey@0"
I20260806 16:17:31.264564 22131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/wal/instance:
uuid: "fcb849e01bf14ac29f38d298b756367a"
format_stamp: "Formatted at 2026-08-06 16:17:31 on dist-test-slave-t3q3"
server_key: "3700d3f890b598c8c0f58e5ece24b0c6"
server_key_iv: "b6ba7836d8908cc6acd3c2b0686a2382"
server_key_version: "encryptionkey@0"
I20260806 16:17:31.270824 22131 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 16:17:31.275084 22153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:31.277148 22131 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 16:17:31.277374 22131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/wal
uuid: "fcb849e01bf14ac29f38d298b756367a"
format_stamp: "Formatted at 2026-08-06 16:17:31 on dist-test-slave-t3q3"
server_key: "3700d3f890b598c8c0f58e5ece24b0c6"
server_key_iv: "b6ba7836d8908cc6acd3c2b0686a2382"
server_key_version: "encryptionkey@0"
I20260806 16:17:31.277588 22131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:31.290825 22131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:31.295310 22131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:31.295583 22131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:31.305773 22131 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.189:37321
I20260806 16:17:31.306129 22231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.189:37321 every 8 connection(s)
I20260806 16:17:31.307363 22131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/data/info.pb
I20260806 16:17:31.311894 22232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:31.312575 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 22131
I20260806 16:17:31.319079 22232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:17:31 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.245.189@KRBTEST.COM
Aug 06 16:17:31 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033051, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.189@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM
I20260806 16:17:31.332587 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.315369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33868 (local address 127.11.245.189:37321)
0806 16:17:31.316132 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.316153 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.316208 (+ 55us) server_negotiation.cc:407] Connection header received
0806 16:17:31.317092 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.317120 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.317619 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.318119 (+ 500us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:31.319150 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.320519 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.321571 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.322442 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.322630 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.325843 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.325850 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.325867 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.325907 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.328014 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.328691 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.328705 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.328713 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.328913 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.329249 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.329254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.329256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.329882 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.330116 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.330494 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.330894 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":214,"threads_started":1}
I20260806 16:17:31.334403 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.321662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.0.0.1:45226)
0806 16:17:31.322282 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:31.322343 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:31.322508 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:31.323033 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:31.323039 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:31.323588 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:31.323763 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.323769 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.324715 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.324719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.325543 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.325554 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.326162 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.326166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.326376 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.326953 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:31.327005 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:31.329347 (+ 2342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:31.332029 (+ 2682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.332040 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.332049 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.332374 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.332746 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.332749 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.332751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.332849 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.333579 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:31.333586 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:31.333730 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 16:17:31.333897 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 16:17:31.334158 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":189,"threads_started":1}
I20260806 16:17:31.334414 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.321712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45226 (local address 127.11.245.190:37595)
0806 16:17:31.321910 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.321916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.322444 (+ 528us) server_negotiation.cc:407] Connection header received
0806 16:17:31.322645 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.322649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.322707 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.322767 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:31.323919 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.324564 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.325676 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.326013 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.326140 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.329637 (+ 3497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.329642 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.329645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.329682 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.331875 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.332536 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.332539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.332541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.332635 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.332994 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.332999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.333000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.333402 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.333490 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.334029 (+ 539us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.334211 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 16:17:31.336802 22232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:17:31 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033051, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.189@KRBTEST.COM for oryx/127.11.245.189@KRBTEST.COM
I20260806 16:17:31.346933 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.337630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33870 (local address 127.11.245.189:37321)
0806 16:17:31.337760 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.337767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.337784 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:17:31.337939 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.337942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.337990 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.338091 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:31.338548 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.339541 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.340426 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.340673 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.340794 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.343272 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.343275 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.343278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.343300 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.345063 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.345540 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.345546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.345548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.345629 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.345932 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.345937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.345939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.346164 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.346282 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.346571 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.346741 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":60}
I20260806 16:17:31.346933 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.337519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37321 (local address 127.0.0.1:33870)
0806 16:17:31.337708 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:31.337726 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:31.337832 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:31.338106 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:31.338109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:31.338315 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:31.338420 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.338426 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.339679 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.339685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.340308 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.340317 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.340798 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.340801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.340884 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.341553 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:31.341566 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:31.343092 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:31.345190 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.345196 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.345200 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.345434 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.345727 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.345731 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.345733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.345840 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.346267 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:31.346270 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:31.346363 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 16:17:31.346478 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 16:17:31.346674 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 16:17:31.351398 22232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: Bootstrap starting.
I20260806 16:17:31.354997 22232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:31.356168 22232 log.cc:824] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:31.359440 22232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: No bootstrap required, opened a new log
I20260806 16:17:31.363492 22232 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } }
I20260806 16:17:31.363834 22232 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:31.363878 22232 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb849e01bf14ac29f38d298b756367a, State: Initialized, Role: FOLLOWER
I20260806 16:17:31.364677 22232 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } }
I20260806 16:17:31.365428 22240 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } } }
I20260806 16:17:31.365609 22240 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:17:31.366580 22232 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: configured and running, proceeding with master startup.
W20260806 16:17:31.372901 22255 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:17:31.373049 22255 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:17:31.375802 22131 master_runner.cc:428] Detected that this master fcb849e01bf14ac29f38d298b756367a is joining an existing cluster
I20260806 16:17:31.375892 22131 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.245.189:37321
I20260806 16:17:31.389110 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.379291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33880 (local address 127.11.245.189:37321)
0806 16:17:31.379434 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.379439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.379518 (+ 79us) server_negotiation.cc:407] Connection header received
0806 16:17:31.379614 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.379618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.379672 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.379759 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:31.380656 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.381482 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.383035 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.383425 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.383565 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.385015 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.385019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.385022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.385046 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.386960 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.387437 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.387442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.387447 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.387544 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.387950 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.387953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.387954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.388234 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.388316 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.388684 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.388830 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":60}
I20260806 16:17:31.389374 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.378873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45230 (local address 127.11.245.190:37595)
0806 16:17:31.379046 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.379052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.379364 (+ 312us) server_negotiation.cc:407] Connection header received
0806 16:17:31.379622 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.379626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.379685 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.379756 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:31.380680 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.381270 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.382612 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.383052 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.383177 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.385045 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.385049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.385052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.385083 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.387289 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.387919 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.387922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.387924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.388012 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.388362 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.388366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.388368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.388620 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.388693 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.389052 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.389194 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 16:17:31.389766 22264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.378837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37321 (local address 127.0.0.1:33880)
0806 16:17:31.379425 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:31.379437 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:31.379530 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:31.379795 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:31.379797 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:31.380191 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:31.380529 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.380536 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.381621 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.381624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.382907 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.382917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.383692 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.383695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.383774 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.384261 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:31.384280 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:31.384868 (+ 588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:31.387069 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.387072 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.387074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.387320 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.387728 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.387731 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.387733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.387835 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.388384 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:31.388387 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:31.388511 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 16:17:31.389286 (+ 775us) client_negotiation.cc:240] Negotiation successful
0806 16:17:31.389479 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":83,"spinlock_wait_cycles":7296,"thread_start_us":143,"threads_started":1}
I20260806 16:17:31.390113 22263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.378794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.0.0.1:45230)
0806 16:17:31.379259 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:31.379277 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:31.379481 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:31.379933 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:31.379936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:31.380193 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:31.380526 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.380536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.381390 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.381395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.382485 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.382497 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.383195 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.383199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.383357 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.384116 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:31.384131 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:31.384868 (+ 737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:31.387420 (+ 2552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.387424 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.387426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.387801 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.388169 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.388171 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.388173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.388261 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.388766 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:31.388769 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:31.388868 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 16:17:31.389600 (+ 732us) client_negotiation.cc:240] Negotiation successful
0806 16:17:31.389748 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":100,"spinlock_wait_cycles":9088,"thread_start_us":155,"threads_started":1}
WARNING: no policy specified for oryx/127.11.245.188@KRBTEST.COM; defaulting to no policy
I20260806 16:17:31.402782 21747 heartbeater.cc:503] Master 127.11.245.190:37595 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.245.188@KRBTEST.COM" created.
Aug 06 16:17:31 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033049, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.190@KRBTEST.COM for oryx/127.11.245.189@KRBTEST.COM
I20260806 16:17:31.411273 22267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.397632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37321 (local address 127.0.0.1:33896)
0806 16:17:31.398279 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:31.398362 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:31.398656 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:31.399105 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:31.399112 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:31.399853 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:31.400052 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.400108 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.401284 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.401288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.402388 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.402401 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.403350 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.403356 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.403508 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.404319 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:31.404336 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:31.406750 (+ 2414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:31.409197 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.409211 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.409220 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.409520 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.409836 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.409840 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.409842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.409934 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.410470 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:31.410481 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:31.410640 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 16:17:31.410823 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 16:17:31.411098 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":165,"threads_started":1}
I20260806 16:17:31.411266 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.397652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33896 (local address 127.11.245.189:37321)
0806 16:17:31.397809 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.397815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.398477 (+ 662us) server_negotiation.cc:407] Connection header received
0806 16:17:31.398854 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.398860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.398942 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.399042 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:31.400247 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.401151 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.402559 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.403018 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.403146 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.406954 (+ 3808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.406962 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.406970 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.407014 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.409063 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.409642 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.409646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.409648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.409736 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.410034 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.410039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.410041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.410336 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.410459 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.410889 (+ 430us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.411053 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 16:17:31.413601 21482 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.245.189:37321: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "fcb849e01bf14ac29f38d298b756367a"
member_type: NON_VOTER
last_known_addr {
host: "127.11.245.189"
port: 37321
}
attrs {
promote: true
}
}
dest_uuid: "4097a09218064c698b4d0b89cffba1ff"
cas_config_opid_index: -1
I20260806 16:17:31.414611 21482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: NON_VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: true } }
I20260806 16:17:31.417634 22273 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER fcb849e01bf14ac29f38d298b756367a (127.11.245.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: NON_VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: true } } }
I20260806 16:17:31.419237 22195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:31.420603 22273 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.245.189:37321
I20260806 16:17:31.420989 22272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4097a09218064c698b4d0b89cffba1ff" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: NON_VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: true } } }
I20260806 16:17:31.421135 22272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:31.421329 22131 master.cc:537] Master@127.11.245.189:37321 shutting down...
I20260806 16:17:31.422484 22195 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 1 FOLLOWER]: Refusing update from remote peer 4097a09218064c698b4d0b89cffba1ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 16:17:31.424422 22272 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: NON_VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:31.427098 22131 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:31.427502 22131 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:31.427656 22131 tablet_replica.cc:330] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: stopping tablet replica
W20260806 16:17:31.428812 21460 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 16:17:31.430358 21460 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff -> Peer fcb849e01bf14ac29f38d298b756367a (127.11.245.189:37321): Couldn't send request to peer fcb849e01bf14ac29f38d298b756367a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.245.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.188.keytab.
Entry for principal oryx/127.11.245.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.188.keytab.
I20260806 16:17:31.432039 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.188:36145
--webserver_interface=127.11.245.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.188.keytab
--principal=oryx/127.11.245.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:37595,127.11.245.189:37321,127.11.245.188:36145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:17:31.434087 22131 master.cc:552] Master@127.11.245.189:37321 shutdown complete.
I20260806 16:17:31.434173 22131 master_runner.cc:305] Clearing existing system tablet
I20260806 16:17:31.434528 22131 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:31.434587 22131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:31.436429 22131 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 16:17:31.437707 22280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:31.438016 22131 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:31.438221 22131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 16:17:31.438287 22131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/wal
uuid: "fcb849e01bf14ac29f38d298b756367a"
format_stamp: "Formatted at 2026-08-06 16:17:31 on dist-test-slave-t3q3"
server_key: "3700d3f890b598c8c0f58e5ece24b0c6"
server_key_iv: "b6ba7836d8908cc6acd3c2b0686a2382"
server_key_version: "encryptionkey@0"
I20260806 16:17:31.441630 22131 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:31.445276 22131 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 16:17:31.445585 22131 log.cc:1196] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 16:17:31.445996 22131 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: Deleting consensus metadata
I20260806 16:17:31.446269 22131 master_runner.cc:315] Copying system tablet from 127.11.245.190:37595
I20260806 16:17:31.447924 22131 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: tablet copy: Beginning tablet copy session from remote peer at address 127.11.245.190:37595
I20260806 16:17:31.459004 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.449018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45242 (local address 127.11.245.190:37595)
0806 16:17:31.449199 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.449207 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.449462 (+ 255us) server_negotiation.cc:407] Connection header received
0806 16:17:31.449723 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.449728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.449796 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.449908 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:31.450517 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.451414 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.452787 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.453162 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.453279 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.454909 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.454914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.454917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.454949 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.456986 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.457537 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.457542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.457544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.457640 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.457946 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.457949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.457951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.458228 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.458332 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.458661 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.458802 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 16:17:31.459024 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.448901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.0.0.1:45242)
0806 16:17:31.449353 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:31.449371 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:31.449585 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:31.449945 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:31.449949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:31.450227 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:31.450381 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.450389 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.451558 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.451561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.452643 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.452658 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.453307 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.453311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.453404 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.454151 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:31.454168 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:31.454724 (+ 556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:31.457122 (+ 2398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.457127 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.457130 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.457420 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.457741 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.457744 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.457747 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.457844 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.458329 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:31.458332 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:31.458457 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 16:17:31.458580 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 16:17:31.458780 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":118,"threads_started":1}
I20260806 16:17:31.460449 21508 tablet_copy_service.cc:140] P 4097a09218064c698b4d0b89cffba1ff: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer fcb849e01bf14ac29f38d298b756367a ({username='oryx', principal='oryx/127.11.245.189@KRBTEST.COM'} at 127.0.0.1:45242)
I20260806 16:17:31.460750 21508 tablet_copy_service.cc:161] P 4097a09218064c698b4d0b89cffba1ff: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer fcb849e01bf14ac29f38d298b756367a at {username='oryx', principal='oryx/127.11.245.189@KRBTEST.COM'} at 127.0.0.1:45242: session id = fcb849e01bf14ac29f38d298b756367a-00000000000000000000000000000000
I20260806 16:17:31.463343 21508 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 16:17:31.466085 22131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 16:17:31.469619 22131 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: tablet copy: Starting download of 0 data blocks...
I20260806 16:17:31.470140 22131 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: tablet copy: Starting download of 1 WAL segments...
I20260806 16:17:31.472834 22131 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 16:17:31.475492 21508 tablet_copy_service.cc:342] P 4097a09218064c698b4d0b89cffba1ff: Request end of tablet copy session fcb849e01bf14ac29f38d298b756367a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.245.189@KRBTEST.COM'} at 127.0.0.1:45242
I20260806 16:17:31.475689 21508 tablet_copy_service.cc:433] P 4097a09218064c698b4d0b89cffba1ff: ending tablet copy session fcb849e01bf14ac29f38d298b756367a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer fcb849e01bf14ac29f38d298b756367a
I20260806 16:17:31.486856 22131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:31.489298 22293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:31.489769 22289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:31.490038 22291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:31.490336 22131 server_base.cc:1034] running on GCE node
I20260806 16:17:31.490658 22131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:31.491061 22131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:31.492182 22131 hybrid_clock.cc:648] HybridClock initialized: now 1786033051492167 us; error 39 us; skew 500 ppm
Aug 06 16:17:31 dist-test-slave-t3q3 krb5kdc[21415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033051, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:31.494164 22131 init.cc:377] Logged in from keytab as oryx/127.11.245.189@KRBTEST.COM (short username oryx)
I20260806 16:17:31.494809 22131 webserver.cc:460] Webserver started at http://127.11.245.189:41985/ using document root <none> and password file <none>
I20260806 16:17:31.495056 22131 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:31.495110 22131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:31.496766 22131 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 16:17:31.497936 22304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:31.498238 22131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 16:17:31.498318 22131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/wal
uuid: "fcb849e01bf14ac29f38d298b756367a"
format_stamp: "Formatted at 2026-08-06 16:17:31 on dist-test-slave-t3q3"
server_key: "3700d3f890b598c8c0f58e5ece24b0c6"
server_key_iv: "b6ba7836d8908cc6acd3c2b0686a2382"
server_key_version: "encryptionkey@0"
I20260806 16:17:31.498411 22131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:31.517480 22131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:31.520398 22131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:31.527163 22131 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.189:37321
I20260806 16:17:31.527189 22381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.189:37321 every 8 connection(s)
I20260806 16:17:31.528254 22131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-1/data/info.pb
I20260806 16:17:31.530812 22383 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:17:31.532173 22383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: Bootstrap starting.
I20260806 16:17:31.546304 22383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:17:31.546773 22383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: Bootstrap complete.
I20260806 16:17:31.547641 22383 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: NON_VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: true } }
I20260806 16:17:31.547835 22383 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: fcb849e01bf14ac29f38d298b756367a, State: Initialized, Role: LEARNER
I20260806 16:17:31.547962 22383 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: NON_VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: true } }
I20260806 16:17:31.548458 22386 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: NON_VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: true } } }
I20260806 16:17:31.548599 22383 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:31.548691 22386 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: This master's current role is: LEARNER
I20260806 16:17:31.553128 22406 catalog_manager.cc:1259] Loaded cluster ID: 36d1ca61ba534096b50a4019dd61d93b
I20260806 16:17:31.553205 22406 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: loading cluster ID for follower catalog manager: success
I20260806 16:17:31.555601 22406 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: acquiring CA information for follower catalog manager: success
I20260806 16:17:31.557322 22406 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 16:17:31.609967 22277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:31.610421 22277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:31.610500 22277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:31.615142 22277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:31.615278 22277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:31.615317 22277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:31.615351 22277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:31.615408 22277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:17:31.623449 22277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:37595,127.11.245.189:37321,127.11.245.188:36145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.188.keytab
--principal=oryx/127.11.245.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.188:36145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.245.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:31.625344 22277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:31.626821 22277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:31.638414 22412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:31.638509 22277 server_base.cc:1034] running on GCE node
W20260806 16:17:31.638414 22415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:31.638415 22413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:31.639833 22277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:31.641510 22277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:31.642711 22277 hybrid_clock.cc:648] HybridClock initialized: now 1786033051642669 us; error 59 us; skew 500 ppm
Aug 06 16:17:31 dist-test-slave-t3q3 krb5kdc[21415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033051, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:31.646435 22277 init.cc:377] Logged in from keytab as oryx/127.11.245.188@KRBTEST.COM (short username oryx)
I20260806 16:17:31.647629 22277 webserver.cc:460] Webserver started at http://127.11.245.188:37945/ using document root <none> and password file <none>
I20260806 16:17:31.648552 22277 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:31.648641 22277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:31.648999 22277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:31.652139 22277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/data/instance:
uuid: "bae5b99b31fb476b8dbf129fb442cd18"
format_stamp: "Formatted at 2026-08-06 16:17:31 on dist-test-slave-t3q3"
server_key: "0f2c4d402381f50c9a6ca5897f975299"
server_key_iv: "e9c7d818e784fa16c7065fb2bde87682"
server_key_version: "encryptionkey@0"
I20260806 16:17:31.652887 22277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/wal/instance:
uuid: "bae5b99b31fb476b8dbf129fb442cd18"
format_stamp: "Formatted at 2026-08-06 16:17:31 on dist-test-slave-t3q3"
server_key: "0f2c4d402381f50c9a6ca5897f975299"
server_key_iv: "e9c7d818e784fa16c7065fb2bde87682"
server_key_version: "encryptionkey@0"
I20260806 16:17:31.658259 22277 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:17:31.661610 22422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:31.663311 22277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:17:31.663468 22277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/wal
uuid: "bae5b99b31fb476b8dbf129fb442cd18"
format_stamp: "Formatted at 2026-08-06 16:17:31 on dist-test-slave-t3q3"
server_key: "0f2c4d402381f50c9a6ca5897f975299"
server_key_iv: "e9c7d818e784fa16c7065fb2bde87682"
server_key_version: "encryptionkey@0"
I20260806 16:17:31.663601 22277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:31.679340 22277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:31.683784 22277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:31.684226 22277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:31.694352 22277 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.188:36145
I20260806 16:17:31.694813 22496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.188:36145 every 8 connection(s)
I20260806 16:17:31.696111 22277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/data/info.pb
I20260806 16:17:31.700871 22498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:31.704938 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 22277
I20260806 16:17:31.707253 22498 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:17:31 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033051, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.188@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM
Aug 06 16:17:31 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.245.188@KRBTEST.COM
I20260806 16:17:31.723058 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.709973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45248 (local address 127.11.245.190:37595)
0806 16:17:31.710120 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.710126 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.710474 (+ 348us) server_negotiation.cc:407] Connection header received
0806 16:17:31.710711 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.710714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.710774 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.710872 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:31.711703 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.712340 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.713535 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.713768 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.713878 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.717004 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.717012 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.717015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.717054 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.719330 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.720191 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.720198 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.720202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.720324 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.720930 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.720934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.720936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.721622 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.721743 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.722636 (+ 893us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.722830 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 16:17:31.724391 22500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.707232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48046 (local address 127.11.245.188:36145)
0806 16:17:31.707867 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.707889 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.707949 (+ 60us) server_negotiation.cc:407] Connection header received
0806 16:17:31.708872 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.708896 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.709415 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.709882 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:31.710941 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.712034 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.713048 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.713586 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.713776 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.717582 (+ 3806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.717594 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.717626 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.717667 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.720379 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.720994 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.721002 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.721004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.721111 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.721546 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.721549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.721551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.722250 (+ 699us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.722438 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.722788 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.723309 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":412,"server-negotiator.queue_time_us":439,"thread_start_us":243,"threads_started":1}
I20260806 16:17:31.724396 22501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.709690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.0.0.1:45248)
0806 16:17:31.710335 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:31.710394 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:31.710583 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:31.710951 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:31.710959 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:31.711344 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:31.711548 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.711561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.712478 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.712489 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.713405 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.713415 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.713958 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.713965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.714066 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.714734 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:31.714753 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:31.716734 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:31.719549 (+ 2815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.719559 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.719574 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.719980 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.720497 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.720501 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.720503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.720756 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.721763 (+ 1007us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:31.721773 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:31.722277 (+ 504us) client_negotiation.cc:769] Sending connection context
0806 16:17:31.722554 (+ 277us) client_negotiation.cc:240] Negotiation successful
0806 16:17:31.723045 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":46,"thread_start_us":192,"threads_started":1}
I20260806 16:17:31.726019 22498 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:17:31.728541 21933 heartbeater.cc:503] Master 127.11.245.190:37595 was elected leader, sending a full tablet report...
Aug 06 16:17:31 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033051, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.188@KRBTEST.COM for oryx/127.11.245.189@KRBTEST.COM
I20260806 16:17:31.737918 22501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.726716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37321 (local address 127.0.0.1:33902)
0806 16:17:31.726886 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:31.726906 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:31.727008 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:31.727653 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:31.727656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:31.727843 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:31.728222 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.728240 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.729506 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.729510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.730161 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.730170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.730770 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.730774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.730909 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.731631 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:31.731645 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:31.733493 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:31.735780 (+ 2287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.735783 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.735785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.736027 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.736630 (+ 603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.736641 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.736643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.736759 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.737358 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:31.737361 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:31.737474 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 16:17:31.737571 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 16:17:31.737732 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 16:17:31.737998 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.726961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33902 (local address 127.11.245.189:37321)
0806 16:17:31.727348 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.727352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.727365 (+ 13us) server_negotiation.cc:407] Connection header received
0806 16:17:31.727442 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.727445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.727494 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.727579 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:31.728434 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.729355 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.730326 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.730610 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.730767 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.733674 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.733678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.733680 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.733712 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.735634 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.736197 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.736201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.736202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.736303 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.736915 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.736919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.736921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.737204 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.737339 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.737699 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.737846 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":194,"threads_started":1}
I20260806 16:17:31.739141 22498 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:17:31 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033051, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.188@KRBTEST.COM for oryx/127.11.245.188@KRBTEST.COM
I20260806 16:17:31.750216 22501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.740157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36145 (local address 127.0.0.1:48062)
0806 16:17:31.740471 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:31.740494 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:31.740649 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:31.740979 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:31.740982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:31.741184 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:31.741284 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.741290 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.742188 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.742191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.742750 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.742759 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.743290 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.743294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.743364 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.744004 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:31.744019 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:31.745860 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:31.747930 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.748234 (+ 304us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.748241 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.748495 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.748831 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.748833 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.748834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.748916 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.749559 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:31.749562 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:31.749663 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 16:17:31.749816 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 16:17:31.750025 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 16:17:31.750243 22500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.740312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48062 (local address 127.11.245.188:36145)
0806 16:17:31.740471 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.740477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.740579 (+ 102us) server_negotiation.cc:407] Connection header received
0806 16:17:31.740814 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.740817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.740859 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.740920 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:31.741416 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.742044 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.742881 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.743137 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.743235 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.746073 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.746077 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.746079 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.746104 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.747783 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.748630 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.748633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.748635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.748731 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.749112 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.749118 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.749121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.749432 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.749529 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.749982 (+ 453us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.750108 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 16:17:31.755703 22498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: Bootstrap starting.
I20260806 16:17:31.759130 22498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:31.760164 22498 log.cc:824] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:31.762838 22498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: No bootstrap required, opened a new log
I20260806 16:17:31.766883 22498 raft_consensus.cc:348] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } }
I20260806 16:17:31.767184 22498 raft_consensus.cc:374] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:31.767225 22498 raft_consensus.cc:729] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bae5b99b31fb476b8dbf129fb442cd18, State: Initialized, Role: FOLLOWER
I20260806 16:17:31.767978 22498 consensus_queue.cc:260] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } }
I20260806 16:17:31.768906 22504 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } } }
I20260806 16:17:31.769083 22504 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:17:31.769994 22498 sys_catalog.cc:564] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 16:17:31.776677 22523 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:17:31.776810 22523 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:17:31.779147 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 21568
I20260806 16:17:31.794694 22277 master_runner.cc:428] Detected that this master bae5b99b31fb476b8dbf129fb442cd18 is joining an existing cluster
I20260806 16:17:31.794803 22277 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.245.188:36145
I20260806 16:17:31.824193 22500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.811082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.11.245.188:36145)
0806 16:17:31.811424 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.811431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.812531 (+ 1100us) server_negotiation.cc:407] Connection header received
0806 16:17:31.812674 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.812679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.812719 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.812768 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:31.813415 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.814251 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.817057 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.817446 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.817535 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.819842 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.819847 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.819850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.819881 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.821882 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.822505 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.822509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.822511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.822608 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.822961 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.822964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.822966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.823259 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.823373 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.823718 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.823929 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":252}
I20260806 16:17:31.824239 22532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.811082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36145 (local address 127.0.0.1:48074)
0806 16:17:31.812442 (+ 1360us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:31.812459 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:31.812583 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:31.812923 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:31.812926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:31.813173 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:31.813306 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.813314 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.814391 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.814394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.815686 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.815704 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.817761 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.817767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.817925 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.818746 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:31.818763 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:31.819638 (+ 875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:31.822044 (+ 2406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.822051 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.822056 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.822384 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.822715 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.822721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.822726 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.822842 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.823377 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:31.823383 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:31.823513 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 16:17:31.823641 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 16:17:31.823851 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1299,"mutex_wait_us":14,"thread_start_us":179,"threads_started":1}
I20260806 16:17:31.828243 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:39661
--local_ip_for_outbound_sockets=127.11.245.129
--tserver_master_addrs=127.11.245.190:37595
--webserver_port=35749
--webserver_interface=127.11.245.129
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.129.keytab
--principal=oryx/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.245.190:37595,127.11.245.189:37321
--tserver_master_addrs=127.11.245.190:37595,127.11.245.189:37321,127.11.245.188:36145 with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:17:31.843035 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.809521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33906 (local address 127.11.245.189:37321)
0806 16:17:31.809764 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.809771 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.811906 (+ 2135us) server_negotiation.cc:407] Connection header received
0806 16:17:31.812189 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.812196 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.812270 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.812335 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:31.813322 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.814251 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.817041 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.817463 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.817605 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.838500 (+ 20895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.838506 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.838510 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.838554 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.840799 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.841435 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.841450 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.841452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.841562 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.841895 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.841898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.841900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.842163 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.842253 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.842637 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.842821 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 16:17:31.843791 22530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.801087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37321 (local address 127.0.0.1:33906)
0806 16:17:31.811786 (+ 10699us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:31.811806 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:31.812026 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:31.812574 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:31.812578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:31.812947 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:31.813147 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.813159 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.814391 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.814394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.816869 (+ 2475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.816885 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.817771 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.817776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.817961 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.837462 (+ 19501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:31.837485 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:31.838200 (+ 715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:31.840984 (+ 2784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.840991 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.840994 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.841297 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.841684 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.841687 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.841689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.841790 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.842313 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:31.842316 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:31.842428 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 16:17:31.843353 (+ 925us) client_negotiation.cc:240] Negotiation successful
0806 16:17:31.843562 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10574,"mutex_wait_us":1441,"spinlock_wait_cycles":2304,"thread_start_us":908,"threads_started":1}
I20260806 16:17:31.847643 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.811261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45256 (local address 127.11.245.190:37595)
0806 16:17:31.811555 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.811563 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.819430 (+ 7867us) server_negotiation.cc:407] Connection header received
0806 16:17:31.819683 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.819689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.819766 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.819829 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:31.837171 (+ 17342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.838200 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.839974 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.840480 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.840602 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.842608 (+ 2006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.842613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.842616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.842657 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.845411 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.846017 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.846022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.846024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.846136 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.846531 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.846535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.846537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.846820 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.846936 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.847297 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.847465 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187}
I20260806 16:17:31.847656 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.811588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.0.0.1:45256)
0806 16:17:31.819291 (+ 7703us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:31.819314 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:31.819545 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:31.820039 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:31.820044 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:31.836505 (+ 16461us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:31.836829 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.836856 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.838441 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.838447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.839745 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.839770 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.840897 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.840903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.841074 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.841741 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:31.841761 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:31.842357 (+ 596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:31.845581 (+ 3224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.845585 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.845587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.845870 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.846269 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.846273 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.846275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.846381 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.846943 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:31.846948 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:31.847089 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 16:17:31.847209 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 16:17:31.847413 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7580,"thread_start_us":602,"threads_started":1}
Aug 06 16:17:31 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033049, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.190@KRBTEST.COM for oryx/127.11.245.188@KRBTEST.COM
I20260806 16:17:31.866736 22267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.854219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36145 (local address 127.0.0.1:48082)
0806 16:17:31.854424 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:31.854444 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:31.854560 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:31.855071 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:31.855075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:31.855373 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:31.855536 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.855545 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.856827 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.856832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.857863 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.857876 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.858562 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.858567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.858689 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.859416 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:31.859436 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:31.861493 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:31.864237 (+ 2744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.864242 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.864245 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.864657 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.865079 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.865083 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.865086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.865211 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.865854 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:31.865857 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:31.865994 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 16:17:31.866119 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 16:17:31.866487 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 16:17:31.867456 22500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.854505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.11.245.188:36145)
0806 16:17:31.854660 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.854667 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.854686 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:17:31.854817 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.854821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.854899 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.855025 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:31.855704 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.856671 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.858022 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.858403 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.858539 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.861740 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.861746 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.861750 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.861799 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.863895 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.864796 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.864801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.864803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.864951 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.865347 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.865351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.865353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.865684 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.865797 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.867076 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.867248 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 16:17:31.869078 21482 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.245.188:36145: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18"
member_type: NON_VOTER
last_known_addr {
host: "127.11.245.188"
port: 36145
}
attrs {
promote: true
}
}
dest_uuid: "4097a09218064c698b4d0b89cffba1ff"
cas_config_opid_index: 5
I20260806 16:17:31.869940 21482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: NON_VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: true } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } }
I20260806 16:17:31.871507 22272 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bae5b99b31fb476b8dbf129fb442cd18 (127.11.245.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: NON_VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: true } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } } }
I20260806 16:17:31.873767 22272 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.245.188:36145
I20260806 16:17:31.874043 22272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4097a09218064c698b4d0b89cffba1ff" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: NON_VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: true } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } } }
I20260806 16:17:31.874204 22272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:31.874782 22277 master.cc:537] Master@127.11.245.188:36145 shutting down...
I20260806 16:17:31.875449 22457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:31.877655 22457 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 1 FOLLOWER]: Refusing update from remote peer 4097a09218064c698b4d0b89cffba1ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 16:17:31.878275 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.871928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33914 (local address 127.11.245.189:37321)
0806 16:17:31.872259 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.872265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.872283 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:17:31.872377 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.872380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.872487 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.872610 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:31.872950 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.873793 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.875429 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.876223 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.876357 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.876672 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.877977 (+ 1305us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.878106 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226}
I20260806 16:17:31.878794 22267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.871746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37321 (local address 127.0.0.1:33914)
0806 16:17:31.871960 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:31.871978 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:31.872132 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:31.872657 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:31.872660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:31.872682 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:31.872812 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.872820 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.873998 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.874001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.875274 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.875289 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.877566 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.877571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.877713 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.877803 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 16:17:31.878535 (+ 732us) client_negotiation.cc:240] Negotiation successful
0806 16:17:31.878654 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260806 16:17:31.878969 22273 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:31.880251 22330 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 1 LEARNER]: Refusing update from remote peer 4097a09218064c698b4d0b89cffba1ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 16:17:31.881024 22273 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: NON_VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260806 16:17:31.881394 21457 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff -> Peer bae5b99b31fb476b8dbf129fb442cd18 (127.11.245.188:36145): Couldn't send request to peer bae5b99b31fb476b8dbf129fb442cd18. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 16:17:31.883687 22330 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bae5b99b31fb476b8dbf129fb442cd18 (127.11.245.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: NON_VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: true } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } } }
I20260806 16:17:31.887006 22386 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4097a09218064c698b4d0b89cffba1ff. Latest consensus state: current_term: 1 leader_uuid: "4097a09218064c698b4d0b89cffba1ff" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: NON_VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: true } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } } }
I20260806 16:17:31.887198 22386 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: This master's current role is: LEARNER
I20260806 16:17:31.887331 22386 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4097a09218064c698b4d0b89cffba1ff" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: NON_VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: true } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } } }
I20260806 16:17:31.887439 22386 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: This master's current role is: LEARNER
I20260806 16:17:31.890002 22273 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff: attempting to promote NON_VOTER fcb849e01bf14ac29f38d298b756367a to VOTER
I20260806 16:17:31.890697 22273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } }
I20260806 16:17:31.892736 22330 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 1 LEARNER]: Refusing update from remote peer 4097a09218064c698b4d0b89cffba1ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 16:17:31.893042 22277 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:31.893450 22277 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:31.893611 22277 tablet_replica.cc:330] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: stopping tablet replica
W20260806 16:17:31.894207 21457 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff -> Peer bae5b99b31fb476b8dbf129fb442cd18 (127.11.245.188:36145): Couldn't send request to peer bae5b99b31fb476b8dbf129fb442cd18. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 16:17:31.894308 22537 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 16:17:31.896863 22537 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, fcb849e01bf14ac29f38d298b756367a (127.11.245.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } } }
I20260806 16:17:31.897897 22330 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, fcb849e01bf14ac29f38d298b756367a (127.11.245.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } } }
I20260806 16:17:31.901055 22277 master.cc:552] Master@127.11.245.188:36145 shutdown complete.
I20260806 16:17:31.901125 22277 master_runner.cc:305] Clearing existing system tablet
I20260806 16:17:31.901721 22277 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:31.901815 22277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:31.903100 22273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4097a09218064c698b4d0b89cffba1ff" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } } }
I20260806 16:17:31.903263 22273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:31.903131 22272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4097a09218064c698b4d0b89cffba1ff" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } } }
I20260806 16:17:31.903388 22272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:31.903676 22536 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4097a09218064c698b4d0b89cffba1ff" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } } }
I20260806 16:17:31.903909 22536 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:17:31.904011 22277 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:17:31.905109 22539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:31.905364 22277 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:31.905525 22277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 16:17:31.905565 22277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/wal
uuid: "bae5b99b31fb476b8dbf129fb442cd18"
format_stamp: "Formatted at 2026-08-06 16:17:31 on dist-test-slave-t3q3"
server_key: "0f2c4d402381f50c9a6ca5897f975299"
server_key_iv: "e9c7d818e784fa16c7065fb2bde87682"
server_key_version: "encryptionkey@0"
I20260806 16:17:31.907799 22277 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:31.910555 22277 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 16:17:31.910692 22277 log.cc:1196] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 16:17:31.911228 22277 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: Deleting consensus metadata
I20260806 16:17:31.911670 22277 master_runner.cc:315] Copying system tablet from 127.11.245.190:37595
I20260806 16:17:31.913335 22277 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: tablet copy: Beginning tablet copy session from remote peer at address 127.11.245.190:37595
I20260806 16:17:31.923607 22548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.914121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.0.0.1:45262)
0806 16:17:31.914536 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:31.914550 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:31.914724 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:31.915049 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:31.915052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:31.915323 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:31.915440 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.915446 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.916365 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.916370 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.916987 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:31.916997 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.917662 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.917666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:31.917782 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.918593 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:31.918607 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:31.919062 (+ 455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:31.921294 (+ 2232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.921298 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.921301 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.921785 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.922151 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:31.922155 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:31.922158 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:31.922265 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:31.922882 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:31.922885 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:31.923025 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 16:17:31.923160 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 16:17:31.923379 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":114,"threads_started":1}
I20260806 16:17:31.923606 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:31.914229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45262 (local address 127.11.245.190:37595)
0806 16:17:31.914432 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:31.914438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:31.914653 (+ 215us) server_negotiation.cc:407] Connection header received
0806 16:17:31.914870 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:31.914873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:31.914925 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:31.915025 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:31.915580 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.916246 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.917120 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:31.917356 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:31.917483 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:31.919257 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:31.919262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:31.919265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:31.919304 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:31.921151 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.921924 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.921928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.921931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.922038 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:31.922374 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:31.922379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:31.922382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:31.922744 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:31.922906 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:31.923232 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 16:17:31.923379 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 16:17:31.924854 21508 tablet_copy_service.cc:140] P 4097a09218064c698b4d0b89cffba1ff: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bae5b99b31fb476b8dbf129fb442cd18 ({username='oryx', principal='oryx/127.11.245.188@KRBTEST.COM'} at 127.0.0.1:45262)
I20260806 16:17:31.925036 21508 tablet_copy_service.cc:161] P 4097a09218064c698b4d0b89cffba1ff: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bae5b99b31fb476b8dbf129fb442cd18 at {username='oryx', principal='oryx/127.11.245.188@KRBTEST.COM'} at 127.0.0.1:45262: session id = bae5b99b31fb476b8dbf129fb442cd18-00000000000000000000000000000000
I20260806 16:17:31.926407 21508 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 16:17:31.928351 22277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 16:17:31.931455 22277 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: tablet copy: Starting download of 0 data blocks...
I20260806 16:17:31.931730 22277 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: tablet copy: Starting download of 1 WAL segments...
I20260806 16:17:31.933959 22277 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 16:17:31.936581 21508 tablet_copy_service.cc:342] P 4097a09218064c698b4d0b89cffba1ff: Request end of tablet copy session bae5b99b31fb476b8dbf129fb442cd18-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.245.188@KRBTEST.COM'} at 127.0.0.1:45262
I20260806 16:17:31.936715 21508 tablet_copy_service.cc:433] P 4097a09218064c698b4d0b89cffba1ff: ending tablet copy session bae5b99b31fb476b8dbf129fb442cd18-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bae5b99b31fb476b8dbf129fb442cd18
I20260806 16:17:31.945586 22277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:31.947770 22549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:31.948019 22550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:31.948455 22552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:31.949571 22277 server_base.cc:1034] running on GCE node
I20260806 16:17:31.949900 22277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:31.950317 22277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:31.951481 22277 hybrid_clock.cc:648] HybridClock initialized: now 1786033051951461 us; error 45 us; skew 500 ppm
Aug 06 16:17:31 dist-test-slave-t3q3 krb5kdc[21415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033051, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:31.953469 22277 init.cc:377] Logged in from keytab as oryx/127.11.245.188@KRBTEST.COM (short username oryx)
I20260806 16:17:31.954161 22277 webserver.cc:460] Webserver started at http://127.11.245.188:39829/ using document root <none> and password file <none>
I20260806 16:17:31.954401 22277 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:31.954466 22277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:31.956139 22277 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 16:17:31.957335 22561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:31.957624 22277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 16:17:31.957731 22277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/wal
uuid: "bae5b99b31fb476b8dbf129fb442cd18"
format_stamp: "Formatted at 2026-08-06 16:17:31 on dist-test-slave-t3q3"
server_key: "0f2c4d402381f50c9a6ca5897f975299"
server_key_iv: "e9c7d818e784fa16c7065fb2bde87682"
server_key_version: "encryptionkey@0"
I20260806 16:17:31.957863 22277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:31.971959 22277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:31.974511 22277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:31.981164 22277 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.188:36145
I20260806 16:17:31.981179 22631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.188:36145 every 8 connection(s)
I20260806 16:17:31.981995 22277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/master-2/data/info.pb
I20260806 16:17:31.983963 22632 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:17:31.985378 22632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: Bootstrap starting.
I20260806 16:17:32.002573 22632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:17:32.003207 22632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: Bootstrap complete.
I20260806 16:17:32.004209 22632 raft_consensus.cc:348] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } }
I20260806 16:17:32.004534 22632 raft_consensus.cc:729] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bae5b99b31fb476b8dbf129fb442cd18, State: Initialized, Role: LEARNER
I20260806 16:17:32.004726 22632 consensus_queue.cc:260] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } }
I20260806 16:17:32.005420 22638 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } } }
I20260806 16:17:32.005575 22638 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [sys.catalog]: This master's current role is: LEARNER
I20260806 16:17:32.005698 22632 sys_catalog.cc:564] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:32.010291 22652 catalog_manager.cc:1259] Loaded cluster ID: 36d1ca61ba534096b50a4019dd61d93b
I20260806 16:17:32.010385 22652 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: loading cluster ID for follower catalog manager: success
I20260806 16:17:32.013274 22652 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: acquiring CA information for follower catalog manager: success
I20260806 16:17:32.015515 22652 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:17:32.018765 22124 heartbeater.cc:503] Master 127.11.245.190:37595 was elected leader, sending a full tablet report...
W20260806 16:17:32.044641 22534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:32.045044 22534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:32.045130 22534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:32.050395 22534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:32.050519 22534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:32.050625 22534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:17:32.057524 22534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.129.keytab
--principal=oryx/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:39661
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=35749
--tserver_master_addrs=127.11.245.190:37595,127.11.245.189:37321,127.11.245.188:36145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:32.059806 22534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:32.061462 22534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:32.075721 22659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:32.075726 22657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:32.075721 22661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:32.076180 22534 server_base.cc:1034] running on GCE node
I20260806 16:17:32.076990 22534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:32.077870 22534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:32.079099 22534 hybrid_clock.cc:648] HybridClock initialized: now 1786033052079082 us; error 49 us; skew 500 ppm
Aug 06 16:17:32 dist-test-slave-t3q3 krb5kdc[21415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033052, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:32.086836 22534 init.cc:377] Logged in from keytab as oryx/127.11.245.129@KRBTEST.COM (short username oryx)
I20260806 16:17:32.088429 22534 webserver.cc:460] Webserver started at http://127.11.245.129:35749/ using document root <none> and password file <none>
I20260806 16:17:32.089322 22534 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:32.089483 22534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:32.095335 22534 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 16:17:32.098688 22673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:32.100832 22534 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:17:32.100999 22534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "f210e2981cc248e9a53de344a59a08ee"
format_stamp: "Formatted at 2026-08-06 16:17:30 on dist-test-slave-t3q3"
server_key: "6fbe05549491346155d5914caac3789b"
server_key_iv: "ae41b5124f958c4a4a0b5e3ab0934539"
server_key_version: "encryptionkey@0"
I20260806 16:17:32.101788 22534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:32.118162 22534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:32.122715 22534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:32.122937 22534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:32.123862 22534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:32.125231 22534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:32.125295 22534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:32.125349 22534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:32.125372 22534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:32.138433 22534 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:39661
I20260806 16:17:32.138458 22823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:39661 every 8 connection(s)
I20260806 16:17:32.139922 22534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:17:32.145336 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 22534
I20260806 16:17:32.145664 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 21758
Aug 06 16:17:32 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033052, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM
Aug 06 16:17:32 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033052, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.188@KRBTEST.COM
Aug 06 16:17:32 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033052, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.189@KRBTEST.COM
I20260806 16:17:32.172312 22829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.142562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:36139 (local address 127.11.245.188:36145)
0806 16:17:32.143319 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:32.143324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:32.144984 (+ 1660us) server_negotiation.cc:407] Connection header received
0806 16:17:32.152443 (+ 7459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:32.152452 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:32.152590 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:32.152702 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:32.156171 (+ 3469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.157748 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.159733 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.160349 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.160488 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.166743 (+ 6255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:32.166750 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:32.166755 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:32.166802 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:32.169043 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.169689 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.169697 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.169703 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.169823 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.170360 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.170367 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.170373 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.170988 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:32.171342 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:32.171850 (+ 508us) server_negotiation.cc:299] Negotiation successful
0806 16:17:32.172052 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":127,"threads_started":1}
I20260806 16:17:32.172308 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.143559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:39847 (local address 127.11.245.190:37595)
0806 16:17:32.143723 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:32.143728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:32.144790 (+ 1062us) server_negotiation.cc:407] Connection header received
0806 16:17:32.154973 (+ 10183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:32.154981 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:32.155086 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:32.155169 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:32.156176 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.156999 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.159148 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.159525 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.159613 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.165157 (+ 5544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:32.165166 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:32.165173 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:32.165220 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:32.168636 (+ 3416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.169924 (+ 1288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.169931 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.169933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.170044 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.170395 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.170399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.170401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.170720 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:32.170830 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:32.171850 (+ 1020us) server_negotiation.cc:299] Negotiation successful
0806 16:17:32.172049 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 16:17:32.174428 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.142948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:47133 (local address 127.11.245.189:37321)
0806 16:17:32.143132 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:32.143138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:32.144802 (+ 1664us) server_negotiation.cc:407] Connection header received
0806 16:17:32.162672 (+ 17870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:32.162677 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:32.162771 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:32.162847 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:32.163685 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.164751 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.166432 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.166870 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.166977 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.170038 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:32.170045 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:32.170051 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:32.170095 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:32.172097 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.172674 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.172680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.172685 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.172802 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.173235 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.173242 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.173246 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.173546 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:32.173633 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:32.174046 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 16:17:32.174210 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 16:17:32.174873 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:46415
--local_ip_for_outbound_sockets=127.11.245.130
--tserver_master_addrs=127.11.245.190:37595
--webserver_port=36225
--webserver_interface=127.11.245.130
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.130.keytab
--principal=oryx/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.245.190:37595,127.11.245.189:37321
--tserver_master_addrs=127.11.245.190:37595,127.11.245.189:37321,127.11.245.188:36145 with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:17:32.174883 22836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.143244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.129:39847)
0806 16:17:32.144539 (+ 1295us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:32.144619 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:32.154715 (+ 10096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:32.155475 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:32.155480 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:32.155819 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:32.156008 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.156018 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.157234 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.157239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.158998 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.159016 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.159940 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.159946 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.160226 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.161905 (+ 1679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:32.161946 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:32.164734 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:32.168837 (+ 4103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.168848 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.168884 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.169763 (+ 879us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.170174 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.170176 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.170178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.170279 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.170930 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:32.170940 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:32.171469 (+ 529us) client_negotiation.cc:769] Sending connection context
0806 16:17:32.172697 (+ 1228us) client_negotiation.cc:240] Negotiation successful
0806 16:17:32.173292 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1183,"mutex_wait_us":35,"spinlock_wait_cycles":31616,"thread_start_us":216,"threads_started":1}
I20260806 16:17:32.175081 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.143244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37321 (local address 127.11.245.129:47133)
0806 16:17:32.144677 (+ 1433us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:32.144692 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:32.162475 (+ 17783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:32.163066 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:32.163072 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:32.163302 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:32.163502 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.163511 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.164972 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.164979 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.166239 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.166257 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.167218 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.167224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.167356 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.167878 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:32.167897 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:32.169828 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:32.172238 (+ 2410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.172244 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.172248 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.172546 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.172948 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.172954 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.172959 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.173096 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.173716 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:32.173722 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:32.173839 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 16:17:32.174600 (+ 761us) client_negotiation.cc:240] Negotiation successful
0806 16:17:32.174785 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1333,"mutex_wait_us":208,"spinlock_wait_cycles":40832,"thread_start_us":168,"threads_started":1}
I20260806 16:17:32.176438 22832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.143244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36145 (local address 127.11.245.129:36139)
0806 16:17:32.144047 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:32.144875 (+ 828us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:32.151794 (+ 6919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:32.153633 (+ 1839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:32.153654 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:32.154752 (+ 1098us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:32.155855 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.155892 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.158025 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.158032 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.159520 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.159539 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.160753 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.160761 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.160941 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.164351 (+ 3410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:32.164368 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:32.166487 (+ 2119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:32.169198 (+ 2711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.169202 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.169205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.169541 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.169948 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.169951 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.169953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.170224 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.171428 (+ 1204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:32.171431 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:32.171527 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 16:17:32.173146 (+ 1619us) client_negotiation.cc:240] Negotiation successful
0806 16:17:32.173359 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":45,"spinlock_wait_cycles":62592,"thread_start_us":183,"threads_started":1}
I20260806 16:17:32.176824 22825 heartbeater.cc:347] Connected to a master server at 127.11.245.190:37595
I20260806 16:17:32.176853 22826 heartbeater.cc:347] Connected to a master server at 127.11.245.189:37321
I20260806 16:17:32.177165 22826 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:32.177163 22825 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:32.177727 22824 heartbeater.cc:347] Connected to a master server at 127.11.245.188:36145
I20260806 16:17:32.177819 22824 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:32.178089 22825 heartbeater.cc:511] Master 127.11.245.190:37595 requested a full tablet report, sending...
I20260806 16:17:32.178088 22824 heartbeater.cc:511] Master 127.11.245.188:36145 requested a full tablet report, sending...
I20260806 16:17:32.178920 22826 heartbeater.cc:511] Master 127.11.245.189:37321 requested a full tablet report, sending...
I20260806 16:17:32.180156 21482 ts_manager.cc:194] Re-registered known tserver with Master: f210e2981cc248e9a53de344a59a08ee (127.11.245.129:39661)
I20260806 16:17:32.180435 22316 ts_manager.cc:194] Registered new tserver with Master: f210e2981cc248e9a53de344a59a08ee (127.11.245.129:39661)
I20260806 16:17:32.180414 22583 ts_manager.cc:194] Registered new tserver with Master: f210e2981cc248e9a53de344a59a08ee (127.11.245.129:39661)
I20260806 16:17:32.181141 21482 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:39847
W20260806 16:17:32.335225 22837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:32.335649 22837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:32.335703 22837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:32.340531 22837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:32.340641 22837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:32.340716 22837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:17:32.347226 22837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.130.keytab
--principal=oryx/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:46415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=36225
--tserver_master_addrs=127.11.245.190:37595,127.11.245.189:37321,127.11.245.188:36145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:32.348910 22837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:32.350435 22837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:32.364121 22843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:32.364131 22845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:32.364121 22842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:32.364881 22837 server_base.cc:1034] running on GCE node
I20260806 16:17:32.366092 22837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:32.367154 22837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:32.368484 22837 hybrid_clock.cc:648] HybridClock initialized: now 1786033052368460 us; error 45 us; skew 500 ppm
Aug 06 16:17:32 dist-test-slave-t3q3 krb5kdc[21415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033052, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:32.372298 22837 init.cc:377] Logged in from keytab as oryx/127.11.245.130@KRBTEST.COM (short username oryx)
I20260806 16:17:32.373653 22837 webserver.cc:460] Webserver started at http://127.11.245.130:36225/ using document root <none> and password file <none>
I20260806 16:17:32.374511 22837 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:32.374660 22837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:32.380551 22837 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:17:32.383433 22856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:32.385389 22837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:17:32.385596 22837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "48485d69c35d4df58077716377f9f008"
format_stamp: "Formatted at 2026-08-06 16:17:30 on dist-test-slave-t3q3"
server_key: "b62f4d6aeaf0b09b9915f767eaf0e1a0"
server_key_iv: "f0297311b78f25905fdc9d6c1a6013a5"
server_key_version: "encryptionkey@0"
I20260806 16:17:32.386358 22837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:32.401443 22837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:32.407219 22837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:32.407490 22837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:32.408509 22837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:32.409771 22837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:32.409852 22837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:32.409927 22837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:32.409981 22837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:32.422911 22837 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:46415
I20260806 16:17:32.422937 23010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:46415 every 8 connection(s)
I20260806 16:17:32.424454 22837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:17:32.427544 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 22837
I20260806 16:17:32.427786 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 21941
Aug 06 16:17:32 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033052, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM
Aug 06 16:17:32 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033052, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.189@KRBTEST.COM
Aug 06 16:17:32 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033052, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.188@KRBTEST.COM
I20260806 16:17:32.447857 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.431897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:37641 (local address 127.11.245.189:37321)
0806 16:17:32.432075 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:32.432080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:32.432410 (+ 330us) server_negotiation.cc:407] Connection header received
0806 16:17:32.432613 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:32.432619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:32.432680 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:32.432734 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:32.433518 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.434460 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.437178 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.437538 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.437636 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.442579 (+ 4943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:32.442585 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:32.442590 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:32.442639 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:32.444622 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.445493 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.445498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.445501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.445607 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.446106 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.446111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.446114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.446401 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:32.446498 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:32.447486 (+ 988us) server_negotiation.cc:299] Negotiation successful
0806 16:17:32.447652 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 16:17:32.447857 22829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.426788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:60995 (local address 127.11.245.188:36145)
0806 16:17:32.426989 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:32.426995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:32.428692 (+ 1697us) server_negotiation.cc:407] Connection header received
0806 16:17:32.430573 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:32.430580 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:32.430663 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:32.430737 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:32.436723 (+ 5986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.437577 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.439348 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.439823 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.439919 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.443034 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:32.443041 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:32.443048 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:32.443091 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:32.445028 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.445721 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.445728 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.445732 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.445832 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.446227 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.446233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.446237 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.446562 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:32.446659 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:32.447471 (+ 812us) server_negotiation.cc:299] Negotiation successful
0806 16:17:32.447681 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 16:17:32.448179 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:44719
--local_ip_for_outbound_sockets=127.11.245.131
--tserver_master_addrs=127.11.245.190:37595
--webserver_port=42023
--webserver_interface=127.11.245.131
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.131.keytab
--principal=oryx/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.245.190:37595,127.11.245.189:37321
--tserver_master_addrs=127.11.245.190:37595,127.11.245.189:37321,127.11.245.188:36145 with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:17:32.448843 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.426738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:33681 (local address 127.11.245.190:37595)
0806 16:17:32.426914 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:32.426920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:32.429103 (+ 2183us) server_negotiation.cc:407] Connection header received
0806 16:17:32.430520 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:32.430527 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:32.430605 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:32.430665 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:32.433890 (+ 3225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.434675 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.437085 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.437529 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.437635 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.442135 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:32.442139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:32.442143 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:32.442184 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:32.444994 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.445712 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.445716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.445718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.445811 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.446155 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.446161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.446165 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.446401 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:32.446490 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:32.447752 (+ 1262us) server_negotiation.cc:299] Negotiation successful
0806 16:17:32.447897 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 16:17:32.450354 23018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.427334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.130:33681)
0806 16:17:32.428996 (+ 1662us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:32.429013 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:32.430242 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:32.431162 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:32.431173 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:32.432035 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:32.433338 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.433361 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.434825 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.434831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.436920 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.436936 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.437905 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.437912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.438385 (+ 473us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.439305 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:32.439337 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:32.441865 (+ 2528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:32.445130 (+ 3265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.445135 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.445138 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.445586 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.445921 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.445923 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.445925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.446049 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.446573 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:32.446583 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:32.447119 (+ 536us) client_negotiation.cc:769] Sending connection context
0806 16:17:32.449226 (+ 2107us) client_negotiation.cc:240] Negotiation successful
0806 16:17:32.449392 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1548,"mutex_wait_us":104,"spinlock_wait_cycles":8576,"thread_start_us":170,"threads_started":1}
I20260806 16:17:32.450367 23017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.428700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37321 (local address 127.11.245.130:37641)
0806 16:17:32.432313 (+ 3613us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:32.432329 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:32.432504 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:32.432886 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:32.432891 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:32.433117 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:32.433334 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.433357 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.434721 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.434727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.437028 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.437039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.437905 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.437908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.438907 (+ 999us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.440219 (+ 1312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:32.440235 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:32.442346 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:32.444788 (+ 2442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.444799 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.444831 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.445348 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.445727 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.445730 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.445732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.445985 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.446573 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:32.446583 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:32.447119 (+ 536us) client_negotiation.cc:769] Sending connection context
0806 16:17:32.448170 (+ 1051us) client_negotiation.cc:240] Negotiation successful
0806 16:17:32.448825 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3526,"mutex_wait_us":2080,"spinlock_wait_cycles":20864,"thread_start_us":118,"threads_started":1}
I20260806 16:17:32.451170 23015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.427333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36145 (local address 127.11.245.130:60995)
0806 16:17:32.428504 (+ 1171us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:32.428572 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:32.430242 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:32.431165 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:32.431175 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:32.432032 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:32.433714 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.433726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.437762 (+ 4036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.437769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.439188 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.439202 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.440183 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.440187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.440297 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.440825 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:32.440842 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:32.442829 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:32.445171 (+ 2342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.445177 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.445182 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.445596 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.445945 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.445951 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.445956 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.446096 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.446725 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:32.446730 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:32.447123 (+ 393us) client_negotiation.cc:769] Sending connection context
0806 16:17:32.448109 (+ 986us) client_negotiation.cc:240] Negotiation successful
0806 16:17:32.448728 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":1838,"spinlock_wait_cycles":16384,"thread_start_us":153,"threads_started":1}
I20260806 16:17:32.452310 23013 heartbeater.cc:347] Connected to a master server at 127.11.245.189:37321
I20260806 16:17:32.452409 23012 heartbeater.cc:347] Connected to a master server at 127.11.245.190:37595
I20260806 16:17:32.452593 23013 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:32.452756 23012 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:32.453238 23011 heartbeater.cc:347] Connected to a master server at 127.11.245.188:36145
I20260806 16:17:32.453353 23011 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:32.453490 23013 heartbeater.cc:511] Master 127.11.245.189:37321 requested a full tablet report, sending...
I20260806 16:17:32.453584 23011 heartbeater.cc:511] Master 127.11.245.188:36145 requested a full tablet report, sending...
I20260806 16:17:32.454593 23012 heartbeater.cc:511] Master 127.11.245.190:37595 requested a full tablet report, sending...
I20260806 16:17:32.455212 22316 ts_manager.cc:194] Registered new tserver with Master: 48485d69c35d4df58077716377f9f008 (127.11.245.130:46415)
I20260806 16:17:32.455505 21482 ts_manager.cc:194] Re-registered known tserver with Master: 48485d69c35d4df58077716377f9f008 (127.11.245.130:46415)
I20260806 16:17:32.456053 22583 ts_manager.cc:194] Registered new tserver with Master: 48485d69c35d4df58077716377f9f008 (127.11.245.130:46415)
I20260806 16:17:32.456456 21482 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:33681
I20260806 16:17:32.477931 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.473478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36145 (local address 127.0.0.1:48094)
0806 16:17:32.473819 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:32.473838 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:32.474030 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:32.474406 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:32.474410 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:32.474431 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:32.474587 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.474593 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.475457 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.475462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.476378 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.476389 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.477488 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.477492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.477578 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.477605 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 16:17:32.477671 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 16:17:32.477757 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":112,"threads_started":1}
I20260806 16:17:32.477931 22829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.473641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48094 (local address 127.11.245.188:36145)
0806 16:17:32.473819 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:32.473826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:32.473945 (+ 119us) server_negotiation.cc:407] Connection header received
0806 16:17:32.474174 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:32.474178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:32.474253 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:32.474378 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:32.474712 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.475330 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.476622 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.477331 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.477488 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.477664 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:32.477705 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 16:17:32.477787 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 16:17:32.479019 22586 raft_consensus.cc:1203] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 16:17:32.479574 22638 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4097a09218064c698b4d0b89cffba1ff. Latest consensus state: current_term: 1 leader_uuid: "4097a09218064c698b4d0b89cffba1ff" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } } }
I20260806 16:17:32.479790 22638 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [sys.catalog]: This master's current role is: LEARNER
W20260806 16:17:32.606325 23019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:32.606778 23019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:32.606837 23019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:32.611531 23019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:32.611619 23019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:32.611691 23019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:17:32.617846 23019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.131.keytab
--principal=oryx/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:44719
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=42023
--tserver_master_addrs=127.11.245.190:37595,127.11.245.189:37321,127.11.245.188:36145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:32.619362 23019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:32.620865 23019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:32.633414 23029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:32.633447 23025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:32.633436 23027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:32.633992 23019 server_base.cc:1034] running on GCE node
I20260806 16:17:32.634541 23019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:32.635413 23019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:32.636616 23019 hybrid_clock.cc:648] HybridClock initialized: now 1786033052636594 us; error 35 us; skew 500 ppm
Aug 06 16:17:32 dist-test-slave-t3q3 krb5kdc[21415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033052, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:32.640002 23019 init.cc:377] Logged in from keytab as oryx/127.11.245.131@KRBTEST.COM (short username oryx)
I20260806 16:17:32.641193 23019 webserver.cc:460] Webserver started at http://127.11.245.131:42023/ using document root <none> and password file <none>
I20260806 16:17:32.641942 23019 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:32.642081 23019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:32.647133 23019 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 16:17:32.649994 23044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:32.651757 23019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:32.651922 23019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "974c509eec194b33b2b65b24a2ed72bb"
format_stamp: "Formatted at 2026-08-06 16:17:30 on dist-test-slave-t3q3"
server_key: "3e32af57b3fa02f59136e391b7f5c914"
server_key_iv: "20c55b84a54ff58f03a27633c3cd2281"
server_key_version: "encryptionkey@0"
I20260806 16:17:32.652714 23019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:32.678694 23019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:32.682842 23019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:32.683104 23019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:32.683931 23019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:32.685130 23019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:32.685182 23019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:32.685253 23019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:32.685284 23019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:32.697726 23019 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:44719
I20260806 16:17:32.697890 23200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:44719 every 8 connection(s)
I20260806 16:17:32.699488 23019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:17:32.701356 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 23019
Aug 06 16:17:32 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033052, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.188@KRBTEST.COM
Aug 06 16:17:32 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033052, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.189@KRBTEST.COM
Aug 06 16:17:32 dist-test-slave-t3q3 krb5kdc[21415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033052, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM
I20260806 16:17:32.719237 23215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.703859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45264 (local address 127.11.245.190:37595)
0806 16:17:32.704192 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:32.704199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:32.705595 (+ 1396us) server_negotiation.cc:407] Connection header received
0806 16:17:32.705701 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:32.705704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:32.705764 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:32.705921 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:32.708038 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.709708 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.711168 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.711670 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.711748 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.713885 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:32.713890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:32.713897 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:32.713928 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:32.716554 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.717195 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.717200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.717202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.717308 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.717809 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.717816 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.717822 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.718385 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:32.718559 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:32.718824 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 16:17:32.718975 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":810,"server-negotiator.queue_time_us":249,"thread_start_us":90,"threads_started":1}
I20260806 16:17:32.724380 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.702298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:44303 (local address 127.11.245.190:37595)
0806 16:17:32.702680 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:32.702686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:32.707679 (+ 4993us) server_negotiation.cc:407] Connection header received
0806 16:17:32.707857 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:32.707860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:32.707904 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:32.707965 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:32.708832 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.709606 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.713605 (+ 3999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.714000 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.714075 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.717768 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:32.717771 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:32.717774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:32.717804 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:32.721103 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.722071 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.722074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.722076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.722174 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.722671 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.722675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.722676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.722922 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:32.723019 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:32.723902 (+ 883us) server_negotiation.cc:299] Negotiation successful
0806 16:17:32.724196 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":263}
I20260806 16:17:32.724831 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.702477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:38789 (local address 127.11.245.189:37321)
0806 16:17:32.704042 (+ 1565us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:32.704048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:32.704748 (+ 700us) server_negotiation.cc:407] Connection header received
0806 16:17:32.706154 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:32.706158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:32.706215 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:32.706288 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:32.709093 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.709877 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.712530 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.712917 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.713045 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.718394 (+ 5349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:32.718398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:32.718401 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:32.718444 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:32.721455 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.722854 (+ 1399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.722857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.722859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.722954 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.723300 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.723303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.723305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.723594 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:32.723677 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:32.724537 (+ 860us) server_negotiation.cc:299] Negotiation successful
0806 16:17:32.724668 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1416,"spinlock_wait_cycles":6272}
I20260806 16:17:32.724853 23216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.703971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48104 (local address 127.11.245.188:36145)
0806 16:17:32.705895 (+ 1924us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:32.705901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:32.705918 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:17:32.705971 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:32.705982 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:32.706036 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:32.706153 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:32.708079 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.708897 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.712275 (+ 3378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.714611 (+ 2336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.715134 (+ 523us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.716461 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:32.716465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:32.716468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:32.716497 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:32.721340 (+ 4843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.722030 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.722035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.722037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.722132 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.722482 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.722488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.722492 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.722777 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:32.724394 (+ 1617us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:32.724492 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 16:17:32.724654 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1751,"thread_start_us":153,"threads_started":1}
I20260806 16:17:32.726528 23218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.704294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33924 (local address 127.11.245.189:37321)
0806 16:17:32.705339 (+ 1045us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:32.705343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:32.705358 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:17:32.705526 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:32.705534 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:32.706080 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:32.706177 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:32.707657 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.709542 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.710746 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.711115 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.711214 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.714070 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:32.714074 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:32.714077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:32.714105 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:32.722692 (+ 8587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.723300 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.723303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.723305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.723990 (+ 685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.725057 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.725060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.725061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.725288 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:32.725418 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:32.726237 (+ 819us) server_negotiation.cc:299] Negotiation successful
0806 16:17:32.726372 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":952,"spinlock_wait_cycles":3712,"thread_start_us":112,"threads_started":1}
I20260806 16:17:32.726572 23212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.702698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37321 (local address 127.11.245.131:38789)
0806 16:17:32.704555 (+ 1857us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:32.704627 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:32.705897 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:32.706900 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:32.707035 (+ 135us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:32.708820 (+ 1785us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:32.708947 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.708957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.710035 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.710037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.712374 (+ 2337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.712392 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.713234 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.713238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.713350 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.714634 (+ 1284us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:32.714651 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:32.718244 (+ 3593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:32.721922 (+ 3678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.721929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.721933 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.722207 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.723078 (+ 871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.723082 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.723087 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.723182 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.723740 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:32.723743 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:32.723849 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 16:17:32.725057 (+ 1208us) client_negotiation.cc:240] Negotiation successful
0806 16:17:32.725233 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1749,"mutex_wait_us":1323,"spinlock_wait_cycles":5504,"thread_start_us":800,"threads_started":1}
I20260806 16:17:32.726913 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.702613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.131:44303)
0806 16:17:32.707581 (+ 4968us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:32.707598 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:32.707754 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:32.708139 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:32.708142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:32.708341 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:32.708666 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.708688 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.709889 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.709893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.711376 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.711386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.714236 (+ 2850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.714241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.714356 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.715319 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:32.715343 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:32.717559 (+ 2216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:32.721317 (+ 3758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.721330 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.721362 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.721947 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.722285 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.722288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.722290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.722557 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.723111 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:32.723120 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:32.723566 (+ 446us) client_negotiation.cc:769] Sending connection context
0806 16:17:32.724628 (+ 1062us) client_negotiation.cc:240] Negotiation successful
0806 16:17:32.725114 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4856,"mutex_wait_us":1447,"thread_start_us":141,"threads_started":1}
I20260806 16:17:32.728009 23202 heartbeater.cc:347] Connected to a master server at 127.11.245.190:37595
I20260806 16:17:32.728360 23202 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:32.729162 23202 heartbeater.cc:511] Master 127.11.245.190:37595 requested a full tablet report, sending...
I20260806 16:17:32.729636 22829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.702319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:53125 (local address 127.11.245.188:36145)
0806 16:17:32.702485 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:32.702490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:32.704748 (+ 2258us) server_negotiation.cc:407] Connection header received
0806 16:17:32.706376 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:32.706379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:32.706435 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:32.706496 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:32.710256 (+ 3760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.711037 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.712189 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.712565 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.712668 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.717927 (+ 5259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:32.717934 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:32.717939 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:32.717970 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:32.721784 (+ 3814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.725577 (+ 3793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.725581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.725583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.725685 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:32.728076 (+ 2391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:32.728098 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:32.728100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:32.728464 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:32.728601 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:32.729172 (+ 571us) server_negotiation.cc:299] Negotiation successful
0806 16:17:32.729391 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":65}
I20260806 16:17:32.729979 23203 heartbeater.cc:347] Connected to a master server at 127.11.245.189:37321
I20260806 16:17:32.730082 23203 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:32.730298 23203 heartbeater.cc:511] Master 127.11.245.189:37321 requested a full tablet report, sending...
I20260806 16:17:32.723439 21482 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 16:17:32.731359 21481 ts_manager.cc:194] Re-registered known tserver with Master: 974c509eec194b33b2b65b24a2ed72bb (127.11.245.131:44719)
I20260806 16:17:32.731626 22323 ts_manager.cc:194] Registered new tserver with Master: 974c509eec194b33b2b65b24a2ed72bb (127.11.245.131:44719)
I20260806 16:17:32.732295 21481 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:44303
W20260806 16:17:32.733372 21482 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:17:32.734807 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.702613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36145 (local address 127.11.245.131:53125)
0806 16:17:32.704555 (+ 1942us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:32.704627 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:32.706249 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:32.706901 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:32.706913 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:32.707761 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:32.708667 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.708688 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.711179 (+ 2491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.711182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.711999 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.712021 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.712741 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.712748 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.712993 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.713717 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:32.713745 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:32.717775 (+ 4030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:32.724006 (+ 6231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.724012 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.724017 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.724338 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.725801 (+ 1463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:32.725804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:32.725806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:32.725901 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:32.728688 (+ 2787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:32.728691 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:32.728856 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 16:17:32.729926 (+ 1070us) client_negotiation.cc:240] Negotiation successful
0806 16:17:32.730214 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1668,"mutex_wait_us":6,"spinlock_wait_cycles":106496,"thread_start_us":237,"threads_started":1}
I20260806 16:17:32.735634 23201 heartbeater.cc:347] Connected to a master server at 127.11.245.188:36145
I20260806 16:17:32.735769 23201 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:32.736048 23201 heartbeater.cc:511] Master 127.11.245.188:36145 requested a full tablet report, sending...
I20260806 16:17:32.737406 22583 ts_manager.cc:194] Registered new tserver with Master: 974c509eec194b33b2b65b24a2ed72bb (127.11.245.131:44719)
I20260806 16:17:32.737859 22537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4097a09218064c698b4d0b89cffba1ff" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: NON_VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: true } } }
I20260806 16:17:32.738034 22537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:32.743026 23224 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff: attempting to promote NON_VOTER bae5b99b31fb476b8dbf129fb442cd18 to VOTER
I20260806 16:17:32.743487 23224 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } }
I20260806 16:17:32.744853 22330 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 1 FOLLOWER]: Refusing update from remote peer 4097a09218064c698b4d0b89cffba1ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 16:17:32.744853 22586 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 1 LEARNER]: Refusing update from remote peer 4097a09218064c698b4d0b89cffba1ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 16:17:32.745524 23224 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 16:17:32.747133 23223 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, bae5b99b31fb476b8dbf129fb442cd18 (127.11.245.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } } }
I20260806 16:17:32.748262 22537 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 16:17:32.748358 22330 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, bae5b99b31fb476b8dbf129fb442cd18 (127.11.245.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } } }
I20260806 16:17:32.749276 23223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4097a09218064c698b4d0b89cffba1ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } } }
I20260806 16:17:32.749394 23223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:32.749423 23224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4097a09218064c698b4d0b89cffba1ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } } }
I20260806 16:17:32.749509 23224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:32.749491 22586 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, bae5b99b31fb476b8dbf129fb442cd18 (127.11.245.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } } }
I20260806 16:17:32.751052 23234 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4097a09218064c698b4d0b89cffba1ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } } }
I20260806 16:17:32.751204 23234 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:17:32.751848 22638 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4097a09218064c698b4d0b89cffba1ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } } }
I20260806 16:17:32.752030 22638 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:17:32.753314 23224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4097a09218064c698b4d0b89cffba1ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } } }
I20260806 16:17:32.753456 23224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:32.761826 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.755093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:39661 (local address 127.0.0.1:59770)
0806 16:17:32.755290 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:32.755311 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:32.755394 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:32.756651 (+ 1257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:32.756655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:32.756672 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:32.756848 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.756856 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.758591 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.758596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.759761 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.759772 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.761314 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.761318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.761448 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.761478 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 16:17:32.761551 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 16:17:32.761640 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260806 16:17:32.761904 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.755372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59770 (local address 127.11.245.129:39661)
0806 16:17:32.755911 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:32.755924 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:32.755951 (+ 27us) server_negotiation.cc:407] Connection header received
0806 16:17:32.756124 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:32.756132 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:32.756465 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:32.757052 (+ 587us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:32.757195 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.758440 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.760047 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.761180 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.761328 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.761538 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:32.761610 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 16:17:32.761737 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":197,"threads_started":1}
I20260806 16:17:32.763617 23236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.755743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:46415 (local address 127.0.0.1:56910)
0806 16:17:32.756440 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:32.756456 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:32.756575 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:32.757568 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:32.757572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:32.757593 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:32.757705 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.757712 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.759418 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.759421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.761163 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.761176 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.763135 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.763139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.763232 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.763257 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 16:17:32.763336 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 16:17:32.763431 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":132,"threads_started":1}
I20260806 16:17:32.764036 23238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.756147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:44719 (local address 127.0.0.1:41436)
0806 16:17:32.758429 (+ 2282us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:32.758441 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:32.758569 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:32.759442 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:32.759446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:32.759466 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:32.759579 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.759586 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.760903 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.760907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.762109 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:32.762120 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.763609 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.763613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:32.763705 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.763729 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 16:17:32.763802 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 16:17:32.763883 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2228,"thread_start_us":92,"threads_started":1}
I20260806 16:17:32.764490 23237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.755975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56910 (local address 127.11.245.130:46415)
0806 16:17:32.756872 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:32.756886 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:32.756911 (+ 25us) server_negotiation.cc:407] Connection header received
0806 16:17:32.757103 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:32.757110 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:32.757316 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:32.757565 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:32.757848 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.759267 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.761336 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.762466 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.762884 (+ 418us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.763075 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:32.763969 (+ 894us) server_negotiation.cc:299] Negotiation successful
0806 16:17:32.764314 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":149,"threads_started":1}
I20260806 16:17:32.764544 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:32.756653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41436 (local address 127.11.245.131:44719)
0806 16:17:32.758303 (+ 1650us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:32.758314 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:32.758544 (+ 230us) server_negotiation.cc:407] Connection header received
0806 16:17:32.758691 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:32.758697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:32.758864 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:32.759262 (+ 398us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:32.759721 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.760735 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.762381 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:32.763248 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:32.763392 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:32.763537 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:32.764260 (+ 723us) server_negotiation.cc:299] Negotiation successful
0806 16:17:32.764382 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1543,"thread_start_us":157,"threads_started":1}
I20260806 16:17:32.766615 22735 tablet_service.cc:1467] Processing CreateTablet for tablet fdafac10c0aa40f58508a3541644c438 (DEFAULT_TABLE table=test-table [id=7a2fe6400312473e8e6baf7ca6dd72d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:32.768290 22735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdafac10c0aa40f58508a3541644c438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:32.767791 22924 tablet_service.cc:1467] Processing CreateTablet for tablet fdafac10c0aa40f58508a3541644c438 (DEFAULT_TABLE table=test-table [id=7a2fe6400312473e8e6baf7ca6dd72d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:32.768994 22924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdafac10c0aa40f58508a3541644c438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:32.770780 23110 tablet_service.cc:1467] Processing CreateTablet for tablet fdafac10c0aa40f58508a3541644c438 (DEFAULT_TABLE table=test-table [id=7a2fe6400312473e8e6baf7ca6dd72d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:32.771883 23110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdafac10c0aa40f58508a3541644c438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:32.779177 23240 tablet_bootstrap.cc:492] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee: Bootstrap starting.
I20260806 16:17:32.780167 23242 tablet_bootstrap.cc:492] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008: Bootstrap starting.
I20260806 16:17:32.781185 23243 tablet_bootstrap.cc:492] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb: Bootstrap starting.
I20260806 16:17:32.783816 23243 tablet_bootstrap.cc:654] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:32.783818 23240 tablet_bootstrap.cc:654] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:32.785053 23243 log.cc:824] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:32.786128 23240 log.cc:824] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:32.786360 23242 tablet_bootstrap.cc:654] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:32.787263 23242 log.cc:824] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:32.787917 23243 tablet_bootstrap.cc:492] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb: No bootstrap required, opened a new log
I20260806 16:17:32.788371 23243 ts_tablet_manager.cc:1397] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260806 16:17:32.788686 23240 tablet_bootstrap.cc:492] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee: No bootstrap required, opened a new log
I20260806 16:17:32.788923 23240 ts_tablet_manager.cc:1397] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260806 16:17:32.790187 23242 tablet_bootstrap.cc:492] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008: No bootstrap required, opened a new log
I20260806 16:17:32.790407 23242 ts_tablet_manager.cc:1397] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260806 16:17:32.792822 23243 raft_consensus.cc:348] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } } peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } }
I20260806 16:17:32.793186 23243 raft_consensus.cc:374] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:32.793246 23243 raft_consensus.cc:729] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 974c509eec194b33b2b65b24a2ed72bb, State: Initialized, Role: FOLLOWER
I20260806 16:17:32.793401 23240 raft_consensus.cc:348] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } } peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } }
I20260806 16:17:32.793718 23240 raft_consensus.cc:374] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:32.793785 23240 raft_consensus.cc:729] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f210e2981cc248e9a53de344a59a08ee, State: Initialized, Role: FOLLOWER
I20260806 16:17:32.793864 23243 consensus_queue.cc:260] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } } peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } }
I20260806 16:17:32.794291 23240 consensus_queue.cc:260] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } } peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } }
I20260806 16:17:32.794870 23202 heartbeater.cc:503] Master 127.11.245.190:37595 was elected leader, sending a full tablet report...
I20260806 16:17:32.795583 23240 ts_tablet_manager.cc:1428] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 16:17:32.795691 23242 raft_consensus.cc:348] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } } peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } }
I20260806 16:17:32.795959 23242 raft_consensus.cc:374] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:32.796034 23242 raft_consensus.cc:729] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48485d69c35d4df58077716377f9f008, State: Initialized, Role: FOLLOWER
I20260806 16:17:32.796569 23242 consensus_queue.cc:260] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } } peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } }
I20260806 16:17:32.799147 23242 ts_tablet_manager.cc:1428] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008: Time spent starting tablet: real 0.009s user 0.003s sys 0.004s
I20260806 16:17:32.799635 23012 heartbeater.cc:503] Master 127.11.245.190:37595 was elected leader, sending a full tablet report...
I20260806 16:17:32.799767 23243 ts_tablet_manager.cc:1428] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20260806 16:17:32.801784 22825 heartbeater.cc:503] Master 127.11.245.190:37595 was elected leader, sending a full tablet report...
W20260806 16:17:32.893167 22828 tablet.cc:2369] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:17:32.927232 23014 tablet.cc:2369] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:17:32.954036 23204 tablet.cc:2369] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:17:33.030750 23253 raft_consensus.cc:482] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:33.031121 23253 raft_consensus.cc:504] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } } peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } }
I20260806 16:17:33.033267 23253 leader_election.cc:288] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f210e2981cc248e9a53de344a59a08ee (127.11.245.129:39661), 974c509eec194b33b2b65b24a2ed72bb (127.11.245.131:44719)
I20260806 16:17:33.037904 23249 raft_consensus.cc:482] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:33.038110 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:33.033706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:38013 (local address 127.11.245.131:44719)
0806 16:17:33.033900 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:33.033907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:33.034191 (+ 284us) server_negotiation.cc:407] Connection header received
0806 16:17:33.034320 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:33.034327 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:33.034394 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:33.034487 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:33.034821 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.035441 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.036766 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.037456 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.037608 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:33.037678 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:33.037872 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 16:17:33.037962 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 16:17:33.038131 23254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:33.033600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:44719 (local address 127.11.245.130:38013)
0806 16:17:33.034055 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:33.034069 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:33.034202 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:33.034507 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:33.034510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:33.034528 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:33.034683 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:33.034691 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.035567 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.035574 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:33.036623 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:33.036634 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.037608 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.037612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:33.037711 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:33.037748 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 16:17:33.037842 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 16:17:33.037936 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":41,"thread_start_us":123,"threads_started":1}
I20260806 16:17:33.038172 23249 raft_consensus.cc:504] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } } peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } }
I20260806 16:17:33.038450 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:33.033741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:35273 (local address 127.11.245.129:39661)
0806 16:17:33.033921 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:33.033927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:33.034376 (+ 449us) server_negotiation.cc:407] Connection header received
0806 16:17:33.034478 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:33.034484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:33.034545 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:33.034634 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:33.034961 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.035477 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.036811 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.037476 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.037614 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:33.037678 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:33.038207 (+ 529us) server_negotiation.cc:299] Negotiation successful
0806 16:17:33.038307 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 16:17:33.038959 23136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdafac10c0aa40f58508a3541644c438" candidate_uuid: "48485d69c35d4df58077716377f9f008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "974c509eec194b33b2b65b24a2ed72bb" is_pre_election: true
I20260806 16:17:33.039250 23255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:33.033599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:39661 (local address 127.11.245.130:35273)
0806 16:17:33.034227 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:33.034245 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:33.034372 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:33.034682 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:33.034685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:33.034701 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:33.034835 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:33.034842 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.035624 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.035627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:33.036640 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:33.036652 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.037771 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.037775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:33.037887 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:33.037914 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 16:17:33.039016 (+ 1102us) client_negotiation.cc:240] Negotiation successful
0806 16:17:33.039107 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":139,"thread_start_us":90,"threads_started":1}
I20260806 16:17:33.039233 23136 raft_consensus.cc:2454] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48485d69c35d4df58077716377f9f008 in term 0.
I20260806 16:17:33.039786 22859 leader_election.cc:302] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48485d69c35d4df58077716377f9f008, 974c509eec194b33b2b65b24a2ed72bb; no voters:
I20260806 16:17:33.039924 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdafac10c0aa40f58508a3541644c438" candidate_uuid: "48485d69c35d4df58077716377f9f008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f210e2981cc248e9a53de344a59a08ee" is_pre_election: true
I20260806 16:17:33.040217 23253 raft_consensus.cc:2781] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:33.040313 23253 raft_consensus.cc:482] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:33.040364 23253 raft_consensus.cc:3037] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:33.040400 22764 raft_consensus.cc:2454] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48485d69c35d4df58077716377f9f008 in term 0.
I20260806 16:17:33.040972 23249 leader_election.cc:288] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f210e2981cc248e9a53de344a59a08ee (127.11.245.129:39661), 48485d69c35d4df58077716377f9f008 (127.11.245.130:46415)
I20260806 16:17:33.042119 23253 raft_consensus.cc:504] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } } peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } }
I20260806 16:17:33.042526 23253 leader_election.cc:288] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [CANDIDATE]: Term 1 election: Requested vote from peers f210e2981cc248e9a53de344a59a08ee (127.11.245.129:39661), 974c509eec194b33b2b65b24a2ed72bb (127.11.245.131:44719)
I20260806 16:17:33.042918 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdafac10c0aa40f58508a3541644c438" candidate_uuid: "48485d69c35d4df58077716377f9f008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f210e2981cc248e9a53de344a59a08ee"
I20260806 16:17:33.043026 22764 raft_consensus.cc:3037] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:33.043793 23136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdafac10c0aa40f58508a3541644c438" candidate_uuid: "48485d69c35d4df58077716377f9f008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "974c509eec194b33b2b65b24a2ed72bb"
I20260806 16:17:33.043967 23136 raft_consensus.cc:3037] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:33.044552 22764 raft_consensus.cc:2454] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48485d69c35d4df58077716377f9f008 in term 1.
I20260806 16:17:33.044896 22862 leader_election.cc:302] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48485d69c35d4df58077716377f9f008, f210e2981cc248e9a53de344a59a08ee; no voters:
I20260806 16:17:33.045006 23212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:33.040148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:46415 (local address 127.11.245.131:46529)
0806 16:17:33.040401 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:33.040418 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:33.040529 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:33.041129 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:33.041135 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:33.041156 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:33.041281 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:33.041287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.042132 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.042137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:33.043453 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:33.043464 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.044506 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.044510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:33.044626 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:33.044654 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 16:17:33.044736 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 16:17:33.044830 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"spinlock_wait_cycles":23552}
I20260806 16:17:33.045028 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:33.040112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:39661 (local address 127.11.245.131:53567)
0806 16:17:33.040395 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:33.040414 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:33.040536 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:33.041420 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:33.041423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:33.041442 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:33.041550 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:33.041555 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.042492 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.042496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:33.043606 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:33.043618 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.044615 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.044621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:33.044723 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:33.044745 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 16:17:33.044812 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 16:17:33.044896 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":1,"spinlock_wait_cycles":17664}
I20260806 16:17:33.045082 23253 raft_consensus.cc:2781] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:33.045323 23253 raft_consensus.cc:686] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [term 1 LEADER]: Becoming Leader. State: Replica: 48485d69c35d4df58077716377f9f008, State: Running, Role: LEADER
I20260806 16:17:33.045418 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:33.040690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:53567 (local address 127.11.245.129:39661)
0806 16:17:33.041197 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:33.041201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:33.041215 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:17:33.041259 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:33.041261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:33.041302 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:33.041372 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:33.041709 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.042367 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.043741 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.044394 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.044510 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:33.044557 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:33.045194 (+ 637us) server_negotiation.cc:299] Negotiation successful
0806 16:17:33.045289 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419}
I20260806 16:17:33.045436 23237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:33.040689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:46529 (local address 127.11.245.130:46415)
0806 16:17:33.040856 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:33.040862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:33.040874 (+ 12us) server_negotiation.cc:407] Connection header received
0806 16:17:33.040921 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:33.040923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:33.040972 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:33.041047 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:33.041415 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.042009 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.043598 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.044222 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.044348 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:33.044433 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:33.045161 (+ 728us) server_negotiation.cc:299] Negotiation successful
0806 16:17:33.045282 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 16:17:33.045643 23136 raft_consensus.cc:2454] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48485d69c35d4df58077716377f9f008 in term 1.
I20260806 16:17:33.045814 23253 consensus_queue.cc:237] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } } peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } }
I20260806 16:17:33.046612 22951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdafac10c0aa40f58508a3541644c438" candidate_uuid: "974c509eec194b33b2b65b24a2ed72bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48485d69c35d4df58077716377f9f008" is_pre_election: true
I20260806 16:17:33.047330 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdafac10c0aa40f58508a3541644c438" candidate_uuid: "974c509eec194b33b2b65b24a2ed72bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f210e2981cc248e9a53de344a59a08ee" is_pre_election: true
I20260806 16:17:33.047533 22764 raft_consensus.cc:2379] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 974c509eec194b33b2b65b24a2ed72bb in current term 1: Already voted for candidate 48485d69c35d4df58077716377f9f008 in this term.
I20260806 16:17:33.048050 23049 leader_election.cc:302] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 974c509eec194b33b2b65b24a2ed72bb; no voters: 48485d69c35d4df58077716377f9f008, f210e2981cc248e9a53de344a59a08ee
I20260806 16:17:33.048395 23249 raft_consensus.cc:2726] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 16:17:33.051028 21482 catalog_manager.cc:5495] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48485d69c35d4df58077716377f9f008 (127.11.245.130). New cstate: current_term: 1 leader_uuid: "48485d69c35d4df58077716377f9f008" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } health_report { overall_health: UNKNOWN } } }
I20260806 16:17:33.136256 23237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:33.131868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56918 (local address 127.11.245.130:46415)
0806 16:17:33.132112 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:33.132123 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:33.132137 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:17:33.132245 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:33.132248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:33.132325 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:33.132463 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:33.132777 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.133366 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.134315 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.134557 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.134688 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:33.135095 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:33.135726 (+ 631us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:33.135895 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:33.135939 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 16:17:33.136012 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 16:17:33.145066 22764 raft_consensus.cc:1261] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee [term 1 FOLLOWER]: Refusing update from remote peer 48485d69c35d4df58077716377f9f008: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:33.145138 23136 raft_consensus.cc:1261] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb [term 1 FOLLOWER]: Refusing update from remote peer 48485d69c35d4df58077716377f9f008: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:33.146123 23256 consensus_queue.cc:1035] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [LEADER]: Connected to new peer: Peer: permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:33.147212 23253 consensus_queue.cc:1035] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:33.164158 23262 mvcc.cc:205] Tried to move back new op lower bound from 7315591385671835648 to 7315591385279479808. Current Snapshot: MvccSnapshot[applied={T|T < 7315591385671835648}]
I20260806 16:17:33.164472 23261 mvcc.cc:205] Tried to move back new op lower bound from 7315591385671835648 to 7315591385279479808. Current Snapshot: MvccSnapshot[applied={T|T < 7315591385671835648 or (T in {7315591385671835648})}]
I20260806 16:17:33.170924 23264 mvcc.cc:205] Tried to move back new op lower bound from 7315591385671835648 to 7315591385279479808. Current Snapshot: MvccSnapshot[applied={T|T < 7315591385671835648}]
I20260806 16:17:33.171497 21482 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:33.171720 21482 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:33.177238 21482 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4097a09218064c698b4d0b89cffba1ff: Sending DeleteTablet for 3 replicas of tablet fdafac10c0aa40f58508a3541644c438
I20260806 16:17:33.178315 22735 tablet_service.cc:1514] Processing DeleteTablet for tablet fdafac10c0aa40f58508a3541644c438 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:33 UTC) from {username='oryx', principal='oryx/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:59770
I20260806 16:17:33.178589 22924 tablet_service.cc:1514] Processing DeleteTablet for tablet fdafac10c0aa40f58508a3541644c438 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:33 UTC) from {username='oryx', principal='oryx/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:56910
I20260806 16:17:33.178678 23110 tablet_service.cc:1514] Processing DeleteTablet for tablet fdafac10c0aa40f58508a3541644c438 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:33 UTC) from {username='oryx', principal='oryx/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:41436
I20260806 16:17:33.179255 23271 tablet_replica.cc:330] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee: stopping tablet replica
I20260806 16:17:33.179730 23271 raft_consensus.cc:2229] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:33.180356 23271 raft_consensus.cc:2258] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:33.181021 23277 tablet_replica.cc:330] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008: stopping tablet replica
I20260806 16:17:33.181275 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 21436
I20260806 16:17:33.191015 23274 tablet_replica.cc:330] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb: stopping tablet replica
I20260806 16:17:33.191697 23274 raft_consensus.cc:2229] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:33.192334 23274 raft_consensus.cc:2258] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:33.195124 23271 ts_tablet_manager.cc:1905] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:33.198336 23277 raft_consensus.cc:2229] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:17:33.199232 23271 ts_tablet_manager.cc:1918] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:33.199378 23271 log.cc:1196] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-0/wal/wals/fdafac10c0aa40f58508a3541644c438
I20260806 16:17:33.199792 23271 ts_tablet_manager.cc:1939] T fdafac10c0aa40f58508a3541644c438 P f210e2981cc248e9a53de344a59a08ee: Deleting consensus metadata
I20260806 16:17:33.200524 23277 raft_consensus.cc:2258] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:33.202069 23274 ts_tablet_manager.cc:1905] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 16:17:33.203661 22675 connection.cc:535] server connection from 127.0.0.1:59770 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 16:17:33.206808 23274 ts_tablet_manager.cc:1918] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:33.206972 23274 log.cc:1196] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/fdafac10c0aa40f58508a3541644c438
I20260806 16:17:33.207404 23274 ts_tablet_manager.cc:1939] T fdafac10c0aa40f58508a3541644c438 P 974c509eec194b33b2b65b24a2ed72bb: Deleting consensus metadata
I20260806 16:17:33.210551 23277 ts_tablet_manager.cc:1905] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:33.215366 23277 ts_tablet_manager.cc:1918] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:33.215559 23277 log.cc:1196] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/fdafac10c0aa40f58508a3541644c438
I20260806 16:17:33.216109 23277 ts_tablet_manager.cc:1939] T fdafac10c0aa40f58508a3541644c438 P 48485d69c35d4df58077716377f9f008: Deleting consensus metadata
I20260806 16:17:33.217903 23218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:33.180183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33936 (local address 127.11.245.189:37321)
0806 16:17:33.180319 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:33.180327 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:33.204838 (+ 24511us) server_negotiation.cc:407] Connection header received
0806 16:17:33.205283 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:33.205288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:33.205366 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:33.205471 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:33.206612 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.207588 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.209348 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.209747 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.209868 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:33.213133 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:33.213142 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:33.213149 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:33.213191 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:33.215433 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:33.216147 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:33.216155 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:33.216158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:33.216276 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:33.216641 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:33.216645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:33.216648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:33.216907 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:33.217069 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:33.217520 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 16:17:33.217700 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 16:17:33.218045 22862 connection.cc:441] server connection from 127.0.0.1:56910 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56910 (request call id 1) could send its response
W20260806 16:17:33.219725 23047 connection.cc:441] server connection from 127.0.0.1:41436 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41436 (request call id 1) could send its response
I20260806 16:17:33.227623 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:33.210028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33938 (local address 127.11.245.189:37321)
0806 16:17:33.210154 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:33.210158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:33.211676 (+ 1518us) server_negotiation.cc:407] Connection header received
0806 16:17:33.212987 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:33.212991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:33.213072 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:33.213128 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:33.216293 (+ 3165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.217116 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.220586 (+ 3470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.220974 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.221052 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:33.223558 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:33.223563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:33.223566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:33.223596 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:33.225368 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:33.225947 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:33.225952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:33.225954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:33.226057 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:33.226356 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:33.226359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:33.226361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:33.226614 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:33.226746 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:33.227122 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 16:17:33.227397 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 16:17:33.228395 23216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:33.209269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48110 (local address 127.11.245.188:36145)
0806 16:17:33.209429 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:33.209434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:33.211309 (+ 1875us) server_negotiation.cc:407] Connection header received
0806 16:17:33.211781 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:33.211785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:33.211856 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:33.211915 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:33.214324 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.215255 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.219293 (+ 4038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.219731 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.219829 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:33.223197 (+ 3368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:33.223201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:33.223205 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:33.223239 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:33.225406 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:33.226047 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:33.226051 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:33.226052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:33.226183 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:33.226747 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:33.226751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:33.226753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:33.227043 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:33.227190 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:33.227611 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 16:17:33.228252 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61,"spinlock_wait_cycles":3968}
I20260806 16:17:33.230862 22829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:33.179852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48106 (local address 127.11.245.188:36145)
0806 16:17:33.180014 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:33.180020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:33.220430 (+ 40410us) server_negotiation.cc:407] Connection header received
0806 16:17:33.220613 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:33.220618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:33.220681 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:33.220744 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:33.221617 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.222219 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.223101 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:33.223467 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:33.224044 (+ 577us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:33.226040 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:33.226044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:33.226047 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:33.226096 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:33.227984 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:33.229307 (+ 1323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:33.229315 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:33.229320 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:33.229420 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:33.229760 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:33.229763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:33.229765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:33.230008 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:33.230150 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:33.230525 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 16:17:33.230657 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41,"spinlock_wait_cycles":4352}
I20260806 16:17:34.166069 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.165305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.130:43239)
0806 16:17:34.165729 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.165890 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":117,"threads_started":1}
W20260806 16:17:34.166661 23012 heartbeater.cc:644] Failed to heartbeat to 127.11.245.190:37595 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
I20260806 16:17:34.166922 23290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.166189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.129:40521)
0806 16:17:34.166553 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.166665 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":93,"threads_started":1}
W20260806 16:17:34.167387 22825 heartbeater.cc:644] Failed to heartbeat to 127.11.245.190:37595 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
I20260806 16:17:34.169734 23292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.169264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.131:39809)
0806 16:17:34.169546 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.169639 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":123,"threads_started":1}
W20260806 16:17:34.170029 23202 heartbeater.cc:644] Failed to heartbeat to 127.11.245.190:37595 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
I20260806 16:17:34.678399 23295 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4097a09218064c698b4d0b89cffba1ff)
I20260806 16:17:34.678323 23294 raft_consensus.cc:482] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4097a09218064c698b4d0b89cffba1ff)
I20260806 16:17:34.678867 23295 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } }
I20260806 16:17:34.678877 23294 raft_consensus.cc:504] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } }
I20260806 16:17:34.680378 23294 leader_election.cc:288] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4097a09218064c698b4d0b89cffba1ff (127.11.245.190:37595), fcb849e01bf14ac29f38d298b756367a (127.11.245.189:37321)
I20260806 16:17:34.680869 23295 leader_election.cc:288] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4097a09218064c698b4d0b89cffba1ff (127.11.245.190:37595), bae5b99b31fb476b8dbf129fb442cd18 (127.11.245.188:36145)
I20260806 16:17:34.681342 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.680599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.0.0.1:45742)
0806 16:17:34.681078 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.681180 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":29,"thread_start_us":98,"threads_started":1}
W20260806 16:17:34.681584 22309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
I20260806 16:17:34.681972 23296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.680595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.0.0.1:45744)
0806 16:17:34.681312 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.681533 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":96,"threads_started":1}
W20260806 16:17:34.682276 22566 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
I20260806 16:17:34.682649 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.682371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.0.0.1:45764)
0806 16:17:34.682522 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.682573 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260806 16:17:34.682780 22309 leader_election.cc:334] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4097a09218064c698b4d0b89cffba1ff (127.11.245.190:37595): Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
I20260806 16:17:34.683365 23296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.683077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.0.0.1:45778)
0806 16:17:34.683224 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.683281 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260806 16:17:34.683480 22566 leader_election.cc:334] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4097a09218064c698b4d0b89cffba1ff (127.11.245.190:37595): Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
I20260806 16:17:34.685761 23299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.680595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36145 (local address 127.0.0.1:48112)
0806 16:17:34.681023 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.681044 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:34.681200 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:34.681730 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:34.681734 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:34.681756 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:34.681930 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:34.681944 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.683247 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.683251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:34.684549 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:34.684562 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.685302 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.685306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:34.685430 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:34.685477 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 16:17:34.685555 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 16:17:34.685632 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":87,"threads_started":1}
I20260806 16:17:34.685801 23301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.680841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48112 (local address 127.11.245.188:36145)
0806 16:17:34.681093 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:34.681100 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:34.681122 (+ 22us) server_negotiation.cc:407] Connection header received
0806 16:17:34.681391 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:34.681396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:34.681467 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:34.681564 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:34.682092 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.683091 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.684692 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.685165 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.685270 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:34.685320 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:34.685591 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 16:17:34.685677 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":166,"thread_start_us":70,"threads_started":1}
I20260806 16:17:34.686470 22586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fcb849e01bf14ac29f38d298b756367a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bae5b99b31fb476b8dbf129fb442cd18" is_pre_election: true
I20260806 16:17:34.686831 23303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.681341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37321 (local address 127.0.0.1:33952)
0806 16:17:34.681733 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.681750 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:34.681861 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:34.683093 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:34.683097 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:34.683123 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:34.683261 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:34.683268 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.684194 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.684198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:34.685342 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:34.685353 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.686356 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.686360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:34.686470 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:34.686508 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 16:17:34.686603 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 16:17:34.686692 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":75,"threads_started":1}
I20260806 16:17:34.686802 22586 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcb849e01bf14ac29f38d298b756367a in term 1.
I20260806 16:17:34.686854 23305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.681660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33952 (local address 127.11.245.189:37321)
0806 16:17:34.682780 (+ 1120us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:34.682785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:34.682800 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:17:34.682846 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:34.682849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:34.682902 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:34.683052 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:34.683390 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.684027 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.685507 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.686231 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.686340 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:34.686380 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:34.686663 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 16:17:34.686755 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":95,"threads_started":1}
I20260806 16:17:34.687297 22305 leader_election.cc:302] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bae5b99b31fb476b8dbf129fb442cd18, fcb849e01bf14ac29f38d298b756367a; no voters: 4097a09218064c698b4d0b89cffba1ff
I20260806 16:17:34.687623 23295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 16:17:34.687722 23295 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4097a09218064c698b4d0b89cffba1ff)
I20260806 16:17:34.687598 22346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bae5b99b31fb476b8dbf129fb442cd18" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fcb849e01bf14ac29f38d298b756367a" is_pre_election: true
I20260806 16:17:34.687767 23295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 1 FOLLOWER]: Advancing to term 2
I20260806 16:17:34.689272 23295 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } }
I20260806 16:17:34.689653 22346 raft_consensus.cc:2379] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bae5b99b31fb476b8dbf129fb442cd18 in current term 2: Already voted for candidate fcb849e01bf14ac29f38d298b756367a in this term.
I20260806 16:17:34.689922 23295 leader_election.cc:288] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [CANDIDATE]: Term 2 election: Requested vote from peers 4097a09218064c698b4d0b89cffba1ff (127.11.245.190:37595), bae5b99b31fb476b8dbf129fb442cd18 (127.11.245.188:36145)
I20260806 16:17:34.690440 23299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.690103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.0.0.1:45792)
0806 16:17:34.690314 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.690378 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260806 16:17:34.690428 22565 leader_election.cc:302] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bae5b99b31fb476b8dbf129fb442cd18; no voters: 4097a09218064c698b4d0b89cffba1ff, fcb849e01bf14ac29f38d298b756367a
I20260806 16:17:34.690997 23294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 1 FOLLOWER]: Advancing to term 2
I20260806 16:17:34.691295 23299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.691069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.0.0.1:45804)
0806 16:17:34.691189 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.691244 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 16:17:34.691478 22309 leader_election.cc:334] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4097a09218064c698b4d0b89cffba1ff (127.11.245.190:37595): Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
I20260806 16:17:34.692214 22586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fcb849e01bf14ac29f38d298b756367a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bae5b99b31fb476b8dbf129fb442cd18"
I20260806 16:17:34.692615 23294 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:17:34.693796 22586 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb849e01bf14ac29f38d298b756367a in term 2.
I20260806 16:17:34.694157 22305 leader_election.cc:302] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bae5b99b31fb476b8dbf129fb442cd18, fcb849e01bf14ac29f38d298b756367a; no voters: 4097a09218064c698b4d0b89cffba1ff
I20260806 16:17:34.694355 23295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 2 FOLLOWER]: Leader election won for term 2
I20260806 16:17:34.694624 23295 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [term 2 LEADER]: Becoming Leader. State: Replica: fcb849e01bf14ac29f38d298b756367a, State: Running, Role: LEADER
I20260806 16:17:34.694993 23295 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } }
I20260806 16:17:34.696484 23309 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcb849e01bf14ac29f38d298b756367a. Latest consensus state: current_term: 2 leader_uuid: "fcb849e01bf14ac29f38d298b756367a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } } }
I20260806 16:17:34.696636 23309 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:34.697000 23312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:34.700702 23312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:34.701246 23312 catalog_manager.cc:1259] Loaded cluster ID: 36d1ca61ba534096b50a4019dd61d93b
I20260806 16:17:34.701298 23312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:34.701978 23312 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:34.702399 23312 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: Loaded TSK: 0
I20260806 16:17:34.703348 23312 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:17:34.720016 22328 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:17:34.724567 22328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:17:34.729405 23299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.728923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.0.0.1:45828)
0806 16:17:34.729163 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.729248 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260806 16:17:34.729696 22586 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [term 2 FOLLOWER]: Refusing update from remote peer fcb849e01bf14ac29f38d298b756367a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 16:17:34.730584 23299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.730183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.0.0.1:45838)
0806 16:17:34.730413 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.730482 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260806 16:17:34.730597 23309 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 16:17:34.730861 22309 consensus_peers.cc:487] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a -> Peer 4097a09218064c698b4d0b89cffba1ff (127.11.245.190:37595): Couldn't send request to peer 4097a09218064c698b4d0b89cffba1ff. Status: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 16:17:34.733834 23294 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcb849e01bf14ac29f38d298b756367a. Latest consensus state: current_term: 2 leader_uuid: "fcb849e01bf14ac29f38d298b756367a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } } }
I20260806 16:17:34.734023 23294 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:17:34.735702 23309 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "fcb849e01bf14ac29f38d298b756367a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } } }
I20260806 16:17:34.735901 23309 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:34.737413 23294 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "fcb849e01bf14ac29f38d298b756367a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4097a09218064c698b4d0b89cffba1ff" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 37595 } } peers { permanent_uuid: "fcb849e01bf14ac29f38d298b756367a" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37321 } attrs { promote: false } } peers { permanent_uuid: "bae5b99b31fb476b8dbf129fb442cd18" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36145 } attrs { promote: false } } }
I20260806 16:17:34.737572 23294 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae5b99b31fb476b8dbf129fb442cd18 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:17:34.754696 23299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.749657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:44719 (local address 127.0.0.1:41442)
0806 16:17:34.749799 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.749818 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:34.749948 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:34.750627 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:34.750630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:34.750650 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:34.750805 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:34.750812 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.752047 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.752051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:34.753328 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:34.753336 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.754210 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.754214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:34.754310 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:34.754360 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 16:17:34.754416 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 16:17:34.754511 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":21}
I20260806 16:17:34.755384 23320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.749730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41442 (local address 127.11.245.131:44719)
0806 16:17:34.750294 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:34.750300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:34.750318 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:17:34.750390 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:34.750394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:34.750464 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:34.750571 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:34.750986 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.751904 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.753450 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.754062 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.754992 (+ 930us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:34.755058 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:34.755103 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 16:17:34.755199 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":172,"threads_started":1}
I20260806 16:17:34.756090 23110 tablet_service.cc:1467] Processing CreateTablet for tablet 7df83f03d7ab48908412f8d5332eb525 (DEFAULT_TABLE table=test-table [id=c362b1c9538e454f87cb60868ce34ba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:34.756191 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.750205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:39661 (local address 127.0.0.1:59772)
0806 16:17:34.750376 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.750390 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:34.750472 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:34.751646 (+ 1174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:34.751649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:34.751670 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:34.751782 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:34.751788 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.752902 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.752905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:34.754208 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:34.754219 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.755565 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.755568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:34.755791 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:34.755848 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 16:17:34.755916 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 16:17:34.756036 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":120}
I20260806 16:17:34.756323 23321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.750390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59772 (local address 127.11.245.129:39661)
0806 16:17:34.750677 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:34.750682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:34.750697 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:17:34.750760 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:34.750762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:34.750814 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:34.750937 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:34.751924 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.752781 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.754327 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.755291 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.755637 (+ 346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:34.755715 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:34.756023 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 16:17:34.756156 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":112,"threads_started":1}
I20260806 16:17:34.756547 23110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7df83f03d7ab48908412f8d5332eb525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:34.756736 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.751397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56924 (local address 127.11.245.130:46415)
0806 16:17:34.751689 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:34.751694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:34.751709 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:17:34.751774 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:34.751777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:34.751838 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:34.751929 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:34.752386 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.753095 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.754664 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.755279 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.755429 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:34.755502 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:34.756486 (+ 984us) server_negotiation.cc:299] Negotiation successful
0806 16:17:34.756589 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":117,"threads_started":1}
I20260806 16:17:34.756896 22735 tablet_service.cc:1467] Processing CreateTablet for tablet 7df83f03d7ab48908412f8d5332eb525 (DEFAULT_TABLE table=test-table [id=c362b1c9538e454f87cb60868ce34ba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:34.757094 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.750914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:46415 (local address 127.0.0.1:56924)
0806 16:17:34.751238 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.751252 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:34.751348 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:34.752106 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:34.752112 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:34.752134 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:34.752243 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:34.752250 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.753229 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.753233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:34.754520 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:34.754530 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.755565 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.755568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:34.755769 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:34.755843 (+ 74us) client_negotiation.cc:769] Sending connection context
0806 16:17:34.755916 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 16:17:34.756962 (+ 1046us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":78,"thread_start_us":171,"threads_started":1}
I20260806 16:17:34.757269 22735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7df83f03d7ab48908412f8d5332eb525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:34.757735 22924 tablet_service.cc:1467] Processing CreateTablet for tablet 7df83f03d7ab48908412f8d5332eb525 (DEFAULT_TABLE table=test-table [id=c362b1c9538e454f87cb60868ce34ba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:34.758068 22924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7df83f03d7ab48908412f8d5332eb525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:34.759994 23324 tablet_bootstrap.cc:492] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb: Bootstrap starting.
I20260806 16:17:34.761179 23326 tablet_bootstrap.cc:492] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008: Bootstrap starting.
I20260806 16:17:34.761215 23324 tablet_bootstrap.cc:654] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:34.761708 23325 tablet_bootstrap.cc:492] T 7df83f03d7ab48908412f8d5332eb525 P f210e2981cc248e9a53de344a59a08ee: Bootstrap starting.
I20260806 16:17:34.762104 23326 tablet_bootstrap.cc:654] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:34.762517 23325 tablet_bootstrap.cc:654] T 7df83f03d7ab48908412f8d5332eb525 P f210e2981cc248e9a53de344a59a08ee: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:34.763605 23326 tablet_bootstrap.cc:492] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008: No bootstrap required, opened a new log
I20260806 16:17:34.763731 23326 ts_tablet_manager.cc:1397] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:34.763978 23324 tablet_bootstrap.cc:492] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb: No bootstrap required, opened a new log
I20260806 16:17:34.763988 23325 tablet_bootstrap.cc:492] T 7df83f03d7ab48908412f8d5332eb525 P f210e2981cc248e9a53de344a59a08ee: No bootstrap required, opened a new log
I20260806 16:17:34.764118 23324 ts_tablet_manager.cc:1397] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 16:17:34.764118 23325 ts_tablet_manager.cc:1397] T 7df83f03d7ab48908412f8d5332eb525 P f210e2981cc248e9a53de344a59a08ee: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 16:17:34.764323 23326 raft_consensus.cc:348] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } } peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } }
I20260806 16:17:34.764472 23326 raft_consensus.cc:374] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:34.764535 23326 raft_consensus.cc:729] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48485d69c35d4df58077716377f9f008, State: Initialized, Role: FOLLOWER
I20260806 16:17:34.764636 23324 raft_consensus.cc:348] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } } peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } }
I20260806 16:17:34.764636 23325 raft_consensus.cc:348] T 7df83f03d7ab48908412f8d5332eb525 P f210e2981cc248e9a53de344a59a08ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } } peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } }
I20260806 16:17:34.764768 23325 raft_consensus.cc:374] T 7df83f03d7ab48908412f8d5332eb525 P f210e2981cc248e9a53de344a59a08ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:34.764768 23324 raft_consensus.cc:374] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:34.764711 23326 consensus_queue.cc:260] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } } peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } }
I20260806 16:17:34.764814 23325 raft_consensus.cc:729] T 7df83f03d7ab48908412f8d5332eb525 P f210e2981cc248e9a53de344a59a08ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f210e2981cc248e9a53de344a59a08ee, State: Initialized, Role: FOLLOWER
I20260806 16:17:34.764814 23324 raft_consensus.cc:729] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 974c509eec194b33b2b65b24a2ed72bb, State: Initialized, Role: FOLLOWER
I20260806 16:17:34.764946 23324 consensus_queue.cc:260] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } } peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } }
I20260806 16:17:34.764953 23325 consensus_queue.cc:260] T 7df83f03d7ab48908412f8d5332eb525 P f210e2981cc248e9a53de344a59a08ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } } peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } }
I20260806 16:17:34.765192 23326 ts_tablet_manager.cc:1428] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 16:17:34.765316 23324 ts_tablet_manager.cc:1428] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:17:34.765316 23325 ts_tablet_manager.cc:1428] T 7df83f03d7ab48908412f8d5332eb525 P f210e2981cc248e9a53de344a59a08ee: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:17:34.767985 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.766161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.131:44737)
0806 16:17:34.766879 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.767826 (+ 947us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":112,"threads_started":1}
I20260806 16:17:34.768110 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.766127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.130:41327)
0806 16:17:34.767870 (+ 1743us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.767939 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1666,"thread_start_us":138,"threads_started":1}
I20260806 16:17:34.768599 23341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.766871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.129:59503)
0806 16:17:34.768403 (+ 1532us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.768466 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1458,"thread_start_us":135,"threads_started":1}
I20260806 16:17:34.769596 23203 heartbeater.cc:503] Master 127.11.245.189:37321 was elected leader, sending a full tablet report...
I20260806 16:17:34.770242 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.769813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.131:39039)
0806 16:17:34.769909 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.770169 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 16:17:34.769833 23341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.769233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.129:34833)
0806 16:17:34.769695 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.769757 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235}
I20260806 16:17:34.771052 22826 heartbeater.cc:503] Master 127.11.245.189:37321 was elected leader, sending a full tablet report...
I20260806 16:17:34.772269 23013 heartbeater.cc:503] Master 127.11.245.189:37321 was elected leader, sending a full tablet report...
I20260806 16:17:34.772652 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.772379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.130:37477)
0806 16:17:34.772544 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.772588 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260806 16:17:34.836169 23336 raft_consensus.cc:482] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:34.836443 23336 raft_consensus.cc:504] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } } peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } }
I20260806 16:17:34.837206 23336 leader_election.cc:288] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f210e2981cc248e9a53de344a59a08ee (127.11.245.129:39661), 48485d69c35d4df58077716377f9f008 (127.11.245.130:46415)
I20260806 16:17:34.837879 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7df83f03d7ab48908412f8d5332eb525" candidate_uuid: "974c509eec194b33b2b65b24a2ed72bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f210e2981cc248e9a53de344a59a08ee" is_pre_election: true
I20260806 16:17:34.837998 22951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7df83f03d7ab48908412f8d5332eb525" candidate_uuid: "974c509eec194b33b2b65b24a2ed72bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48485d69c35d4df58077716377f9f008" is_pre_election: true
I20260806 16:17:34.838133 22764 raft_consensus.cc:2454] T 7df83f03d7ab48908412f8d5332eb525 P f210e2981cc248e9a53de344a59a08ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 974c509eec194b33b2b65b24a2ed72bb in term 0.
I20260806 16:17:34.838296 22951 raft_consensus.cc:2454] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 974c509eec194b33b2b65b24a2ed72bb in term 0.
I20260806 16:17:34.838657 23049 leader_election.cc:302] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 974c509eec194b33b2b65b24a2ed72bb, f210e2981cc248e9a53de344a59a08ee; no voters:
I20260806 16:17:34.839006 23336 raft_consensus.cc:2781] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:34.839478 23336 raft_consensus.cc:482] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:34.839565 23336 raft_consensus.cc:3037] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:34.841391 23336 raft_consensus.cc:504] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } } peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } }
I20260806 16:17:34.841981 23336 leader_election.cc:288] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [CANDIDATE]: Term 1 election: Requested vote from peers f210e2981cc248e9a53de344a59a08ee (127.11.245.129:39661), 48485d69c35d4df58077716377f9f008 (127.11.245.130:46415)
I20260806 16:17:34.842659 22951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7df83f03d7ab48908412f8d5332eb525" candidate_uuid: "974c509eec194b33b2b65b24a2ed72bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48485d69c35d4df58077716377f9f008"
I20260806 16:17:34.842875 22951 raft_consensus.cc:3037] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:34.844347 22951 raft_consensus.cc:2454] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 974c509eec194b33b2b65b24a2ed72bb in term 1.
I20260806 16:17:34.844323 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7df83f03d7ab48908412f8d5332eb525" candidate_uuid: "974c509eec194b33b2b65b24a2ed72bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f210e2981cc248e9a53de344a59a08ee"
I20260806 16:17:34.844522 22764 raft_consensus.cc:3037] T 7df83f03d7ab48908412f8d5332eb525 P f210e2981cc248e9a53de344a59a08ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:34.844756 23047 leader_election.cc:302] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48485d69c35d4df58077716377f9f008, 974c509eec194b33b2b65b24a2ed72bb; no voters:
I20260806 16:17:34.844980 23336 raft_consensus.cc:2781] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:34.845290 23336 raft_consensus.cc:686] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [term 1 LEADER]: Becoming Leader. State: Replica: 974c509eec194b33b2b65b24a2ed72bb, State: Running, Role: LEADER
I20260806 16:17:34.845786 23336 consensus_queue.cc:237] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } } peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } }
I20260806 16:17:34.846213 22764 raft_consensus.cc:2454] T 7df83f03d7ab48908412f8d5332eb525 P f210e2981cc248e9a53de344a59a08ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 974c509eec194b33b2b65b24a2ed72bb in term 1.
I20260806 16:17:34.847355 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.846919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.131:38335)
0806 16:17:34.847135 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.847243 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 16:17:34.849505 22323 catalog_manager.cc:5495] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 974c509eec194b33b2b65b24a2ed72bb (127.11.245.131). New cstate: current_term: 1 leader_uuid: "974c509eec194b33b2b65b24a2ed72bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "974c509eec194b33b2b65b24a2ed72bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 } health_report { overall_health: UNKNOWN } } }
I20260806 16:17:34.873598 23320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.869313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41458 (local address 127.11.245.131:44719)
0806 16:17:34.869551 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:34.869556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:34.869570 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:17:34.869679 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:34.869681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:34.869753 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:34.869882 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:34.870205 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.870793 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.871721 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:34.871958 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:34.872111 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:34.872448 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:34.873118 (+ 670us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:34.873258 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:34.873319 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 16:17:34.873397 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 16:17:34.877934 22764 raft_consensus.cc:1261] T 7df83f03d7ab48908412f8d5332eb525 P f210e2981cc248e9a53de344a59a08ee [term 1 FOLLOWER]: Refusing update from remote peer 974c509eec194b33b2b65b24a2ed72bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:34.878048 22951 raft_consensus.cc:1261] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008 [term 1 FOLLOWER]: Refusing update from remote peer 974c509eec194b33b2b65b24a2ed72bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:34.878738 23342 consensus_queue.cc:1035] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f210e2981cc248e9a53de344a59a08ee" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 39661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:34.879159 23336 consensus_queue.cc:1035] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "48485d69c35d4df58077716377f9f008" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 46415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:34.881855 23341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.881281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.129:35171)
0806 16:17:34.881498 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.881727 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260806 16:17:34.884544 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.883907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.130:46605)
0806 16:17:34.884152 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.884222 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260806 16:17:34.888829 23341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.888434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.129:43389)
0806 16:17:34.888632 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.888713 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260806 16:17:34.888978 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.888202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.130:55159)
0806 16:17:34.888821 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.888886 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511}
I20260806 16:17:34.891691 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.891295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:37595 (local address 127.11.245.131:41205)
0806 16:17:34.891478 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.891571 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 16:17:34.898239 22316 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:34.898461 22316 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:34.904208 22316 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fcb849e01bf14ac29f38d298b756367a: Sending DeleteTablet for 3 replicas of tablet 7df83f03d7ab48908412f8d5332eb525
I20260806 16:17:34.905313 23110 tablet_service.cc:1514] Processing DeleteTablet for tablet 7df83f03d7ab48908412f8d5332eb525 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:34 UTC) from {username='oryx', principal='oryx/127.11.245.189@KRBTEST.COM'} at 127.0.0.1:41442
I20260806 16:17:34.905441 22735 tablet_service.cc:1514] Processing DeleteTablet for tablet 7df83f03d7ab48908412f8d5332eb525 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:34 UTC) from {username='oryx', principal='oryx/127.11.245.189@KRBTEST.COM'} at 127.0.0.1:59772
I20260806 16:17:34.905639 22924 tablet_service.cc:1514] Processing DeleteTablet for tablet 7df83f03d7ab48908412f8d5332eb525 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:34 UTC) from {username='oryx', principal='oryx/127.11.245.189@KRBTEST.COM'} at 127.0.0.1:56924
I20260806 16:17:34.905982 23361 tablet_replica.cc:330] T 7df83f03d7ab48908412f8d5332eb525 P f210e2981cc248e9a53de344a59a08ee: stopping tablet replica
I20260806 16:17:34.906000 23359 tablet_replica.cc:330] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb: stopping tablet replica
I20260806 16:17:34.906191 23361 raft_consensus.cc:2229] T 7df83f03d7ab48908412f8d5332eb525 P f210e2981cc248e9a53de344a59a08ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:34.906349 23359 raft_consensus.cc:2229] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:17:34.906457 23361 raft_consensus.cc:2258] T 7df83f03d7ab48908412f8d5332eb525 P f210e2981cc248e9a53de344a59a08ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:34.907313 23359 raft_consensus.cc:2258] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:34.907542 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 22534
I20260806 16:17:34.912043 23359 ts_tablet_manager.cc:1905] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:34.915093 23359 ts_tablet_manager.cc:1918] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:34.915230 23359 log.cc:1196] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/7df83f03d7ab48908412f8d5332eb525
I20260806 16:17:34.915604 23359 ts_tablet_manager.cc:1939] T 7df83f03d7ab48908412f8d5332eb525 P 974c509eec194b33b2b65b24a2ed72bb: Deleting consensus metadata
I20260806 16:17:34.916940 22307 catalog_manager.cc:4841] TS 974c509eec194b33b2b65b24a2ed72bb (127.11.245.131:44719): tablet 7df83f03d7ab48908412f8d5332eb525 (table test-table [id=c362b1c9538e454f87cb60868ce34ba3]) successfully deleted
I20260806 16:17:34.921459 23364 tablet_replica.cc:330] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008: stopping tablet replica
I20260806 16:17:34.921720 23364 raft_consensus.cc:2229] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:34.921990 23364 raft_consensus.cc:2258] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:34.925114 23364 ts_tablet_manager.cc:1905] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:34.929788 23364 ts_tablet_manager.cc:1918] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:34.929941 23364 log.cc:1196] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/7df83f03d7ab48908412f8d5332eb525
I20260806 16:17:34.930465 23364 ts_tablet_manager.cc:1939] T 7df83f03d7ab48908412f8d5332eb525 P 48485d69c35d4df58077716377f9f008: Deleting consensus metadata
I20260806 16:17:34.931770 22307 catalog_manager.cc:4841] TS 48485d69c35d4df58077716377f9f008 (127.11.245.130:46415): tablet 7df83f03d7ab48908412f8d5332eb525 (table test-table [id=c362b1c9538e454f87cb60868ce34ba3]) successfully deleted
I20260806 16:17:34.936685 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:34.936303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:39661 (local address 127.0.0.1:59788)
0806 16:17:34.936472 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:34.936556 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:39661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 16:17:34.936677 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 22837
W20260806 16:17:34.951881 22309 catalog_manager.cc:4584] TS f210e2981cc248e9a53de344a59a08ee (127.11.245.129:39661): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7df83f03d7ab48908412f8d5332eb525: Network error: Client connection negotiation failed: client connection to 127.11.245.129:39661: connect: Connection refused (error 111)
I20260806 16:17:34.965489 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 23019
I20260806 16:17:34.986202 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 22131
I20260806 16:17:35.005208 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 22277
2026-08-06T16:17:35Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6718 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:17:35 dist-test-slave-t3q3 krb5kdc[23370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:17:35 dist-test-slave-t3q3 krb5kdc[23370](info): set up 2 sockets
Aug 06 16:17:35 dist-test-slave-t3q3 krb5kdc[23370](info): commencing operation
krb5kdc: starting...
W20260806 16:17:36.309738 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:17:36 dist-test-slave-t3q3 krb5kdc[23370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:17:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:36Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.190.keytab.
Entry for principal oryx/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.190.keytab.
I20260806 16:17:36.463354 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:44551
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.190.keytab
--principal=oryx/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:44551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:36.605708 23386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:36.606151 23386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:36.606235 23386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:36.610708 23386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:36.610831 23386 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 16:17:36.610865 23386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:36.610909 23386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:36.610968 23386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:36.611011 23386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:17:36.617511 23386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:44551
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.190.keytab
--principal=oryx/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:44551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:36.619097 23386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:36.620519 23386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:36.630982 23392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:36.631014 23391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:36.631136 23386 server_base.cc:1034] running on GCE node
W20260806 16:17:36.631335 23396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:36.632195 23386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:36.633630 23386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:36.634835 23386 hybrid_clock.cc:648] HybridClock initialized: now 1786033056634820 us; error 36 us; skew 500 ppm
Aug 06 16:17:36 dist-test-slave-t3q3 krb5kdc[23370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033056, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:36.638221 23386 init.cc:377] Logged in from keytab as oryx/127.11.245.190@KRBTEST.COM (short username oryx)
I20260806 16:17:36.639402 23386 webserver.cc:460] Webserver started at http://127.11.245.190:34891/ using document root <none> and password file <none>
I20260806 16:17:36.640206 23386 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:36.640337 23386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:36.640637 23386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:36.643430 23386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "ba27428000ad492dbb04f5985577ab27"
format_stamp: "Formatted at 2026-08-06 16:17:36 on dist-test-slave-t3q3"
server_key: "086f87c1fd6f1062ed74690cc05afb58"
server_key_iv: "7815aa0d0968a3f8f9b95708dddd2e32"
server_key_version: "encryptionkey@0"
I20260806 16:17:36.644217 23386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "ba27428000ad492dbb04f5985577ab27"
format_stamp: "Formatted at 2026-08-06 16:17:36 on dist-test-slave-t3q3"
server_key: "086f87c1fd6f1062ed74690cc05afb58"
server_key_iv: "7815aa0d0968a3f8f9b95708dddd2e32"
server_key_version: "encryptionkey@0"
I20260806 16:17:36.649180 23386 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:17:36.652872 23405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:36.654600 23386 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:17:36.654779 23386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "ba27428000ad492dbb04f5985577ab27"
format_stamp: "Formatted at 2026-08-06 16:17:36 on dist-test-slave-t3q3"
server_key: "086f87c1fd6f1062ed74690cc05afb58"
server_key_iv: "7815aa0d0968a3f8f9b95708dddd2e32"
server_key_version: "encryptionkey@0"
I20260806 16:17:36.654929 23386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:36.677843 23386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:36.681850 23386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:36.682111 23386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:36.693478 23386 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:44551
I20260806 16:17:36.693478 23495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:44551 every 8 connection(s)
I20260806 16:17:36.694981 23386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:17:36.699168 23496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:36.704201 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 23386
I20260806 16:17:36.704432 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:17:36.704797 12246 external_mini_cluster.cc:1428] Setting key 2245adebd7453a48c75e4326ea70d172
I20260806 16:17:36.708235 23496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27: Bootstrap starting.
I20260806 16:17:36.711783 23496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:36.713079 23496 log.cc:824] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:36.716593 23496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27: No bootstrap required, opened a new log
Aug 06 16:17:36 dist-test-slave-t3q3 krb5kdc[23370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM
I20260806 16:17:36.721990 23496 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba27428000ad492dbb04f5985577ab27" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44551 } }
I20260806 16:17:36.722430 23496 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:36.722493 23496 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba27428000ad492dbb04f5985577ab27, State: Initialized, Role: FOLLOWER
I20260806 16:17:36.723198 23496 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba27428000ad492dbb04f5985577ab27" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44551 } }
I20260806 16:17:36.723418 23496 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:17:36.723498 23496 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:17:36.723613 23496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:36.725092 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:36.707125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38118 (local address 127.11.245.190:44551)
0806 16:17:36.707770 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:36.707790 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:36.707838 (+ 48us) server_negotiation.cc:407] Connection header received
0806 16:17:36.708825 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:36.708851 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:36.709309 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:36.709750 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:36.710887 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:36.712290 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:36.713596 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:36.714518 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:36.715329 (+ 811us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:36.718634 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:36.718640 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:36.718660 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:36.718694 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:36.720969 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:36.721568 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:36.721574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:36.721581 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:36.721723 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:36.722118 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:36.722122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:36.722124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:36.722800 (+ 676us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:36.723060 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:36.723383 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 16:17:36.723794 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":212,"threads_started":1}
I20260806 16:17:36.725169 23496 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba27428000ad492dbb04f5985577ab27" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44551 } }
I20260806 16:17:36.725672 23496 leader_election.cc:302] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba27428000ad492dbb04f5985577ab27; no voters:
I20260806 16:17:36.726109 23496 leader_election.cc:288] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:17:36.726316 23502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:36.726579 23502 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [term 1 LEADER]: Becoming Leader. State: Replica: ba27428000ad492dbb04f5985577ab27, State: Running, Role: LEADER
I20260806 16:17:36.726976 23502 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba27428000ad492dbb04f5985577ab27" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44551 } }
I20260806 16:17:36.727764 23496 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:36.728564 23502 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba27428000ad492dbb04f5985577ab27. Latest consensus state: current_term: 1 leader_uuid: "ba27428000ad492dbb04f5985577ab27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba27428000ad492dbb04f5985577ab27" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44551 } } }
I20260806 16:17:36.728694 23502 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:36.730948 23503 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba27428000ad492dbb04f5985577ab27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba27428000ad492dbb04f5985577ab27" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44551 } } }
I20260806 16:17:36.731109 23503 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27 [sys.catalog]: This master's current role is: LEADER
W20260806 16:17:36.739617 23520 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:17:36.739763 23520 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:17:36.739852 23512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:36.741338 23512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:36.751192 23512 catalog_manager.cc:1347] Generated new cluster ID: a7da043b3d2b49ccac3cc9164ce9ca57
I20260806 16:17:36.751315 23512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:36.776139 23512 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:17:36.777856 23512 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:36.783569 23512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27: Generated new TSK 0
I20260806 16:17:36.784614 23512 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.129.keytab.
Entry for principal oryx/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.129.keytab.
I20260806 16:17:36.831964 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:44551
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.129.keytab
--principal=oryx/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:36.974824 23530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:36.975175 23530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:36.975227 23530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:36.979424 23530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:36.979511 23530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:36.979554 23530 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:17:36.979591 23530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:17:36.985477 23530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.129.keytab
--principal=oryx/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.245.190:44551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:36.986929 23530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:36.988229 23530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:37.000962 23543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:37.000962 23540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:37.001299 23530 server_base.cc:1034] running on GCE node
W20260806 16:17:37.000963 23539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:37.002022 23530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:37.002864 23530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:37.004105 23530 hybrid_clock.cc:648] HybridClock initialized: now 1786033057004013 us; error 48 us; skew 500 ppm
Aug 06 16:17:37 dist-test-slave-t3q3 krb5kdc[23370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033057, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:37.011459 23530 init.cc:377] Logged in from keytab as oryx/127.11.245.129@KRBTEST.COM (short username oryx)
I20260806 16:17:37.012869 23530 webserver.cc:460] Webserver started at http://127.11.245.129:42737/ using document root <none> and password file <none>
I20260806 16:17:37.013651 23530 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:37.013758 23530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:37.014127 23530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:37.017104 23530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "7929ac83634d43659b18bc9402685058"
format_stamp: "Formatted at 2026-08-06 16:17:37 on dist-test-slave-t3q3"
server_key: "1e8242da889e5905a9b707110122ff3b"
server_key_iv: "19ed60d81d6dd701fa613ce473244316"
server_key_version: "encryptionkey@0"
I20260806 16:17:37.017819 23530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "7929ac83634d43659b18bc9402685058"
format_stamp: "Formatted at 2026-08-06 16:17:37 on dist-test-slave-t3q3"
server_key: "1e8242da889e5905a9b707110122ff3b"
server_key_iv: "19ed60d81d6dd701fa613ce473244316"
server_key_version: "encryptionkey@0"
I20260806 16:17:37.022531 23530 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:17:37.025592 23550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:37.027117 23530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 16:17:37.027293 23530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "7929ac83634d43659b18bc9402685058"
format_stamp: "Formatted at 2026-08-06 16:17:37 on dist-test-slave-t3q3"
server_key: "1e8242da889e5905a9b707110122ff3b"
server_key_iv: "19ed60d81d6dd701fa613ce473244316"
server_key_version: "encryptionkey@0"
I20260806 16:17:37.027431 23530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:37.062181 23530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:37.066339 23530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:37.066609 23530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:37.068799 23530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:37.068884 23530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:37.068970 23530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:37.069013 23530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 16:17:37 dist-test-slave-t3q3 krb5kdc[23370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033057, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM
I20260806 16:17:37.083118 23530 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:36611
I20260806 16:17:37.083207 23706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:36611 every 8 connection(s)
I20260806 16:17:37.084578 23530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:17:37.086557 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.069876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:52043 (local address 127.11.245.190:44551)
0806 16:17:37.070046 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:37.070051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:37.071367 (+ 1316us) server_negotiation.cc:407] Connection header received
0806 16:17:37.072666 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:37.072672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:37.072758 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:37.072834 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:37.074862 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.076043 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.077413 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.077873 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.078020 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.081457 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:37.081464 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:37.081472 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:37.081514 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:37.083481 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:37.084033 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:37.084040 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:37.084045 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:37.084183 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:37.084648 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:37.084654 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:37.084656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:37.085057 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:37.085195 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:37.086179 (+ 984us) server_negotiation.cc:299] Negotiation successful
0806 16:17:37.086382 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 16:17:37.087447 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.070539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44551 (local address 127.11.245.129:52043)
0806 16:17:37.071181 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:37.071232 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:37.072348 (+ 1116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:37.073434 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:37.073443 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:37.073841 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:37.074670 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.074697 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.076243 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.076247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.077252 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.077262 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.078107 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.078113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.078369 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.079166 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:37.079192 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:37.081244 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:37.083604 (+ 2360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:37.083609 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:37.083624 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:37.083914 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:37.084309 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:37.084316 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:37.084319 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:37.084522 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:37.085352 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:37.085361 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:37.085737 (+ 376us) client_negotiation.cc:769] Sending connection context
0806 16:17:37.085983 (+ 246us) client_negotiation.cc:240] Negotiation successful
0806 16:17:37.086332 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":182,"threads_started":1}
I20260806 16:17:37.089025 23707 heartbeater.cc:347] Connected to a master server at 127.11.245.190:44551
I20260806 16:17:37.089334 23707 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:37.090078 23707 heartbeater.cc:511] Master 127.11.245.190:44551 requested a full tablet report, sending...
I20260806 16:17:37.092454 23427 ts_manager.cc:194] Registered new tserver with Master: 7929ac83634d43659b18bc9402685058 (127.11.245.129:36611)
I20260806 16:17:37.093101 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 23530
I20260806 16:17:37.093276 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:17:37.093705 12246 external_mini_cluster.cc:1428] Setting key 34a868f0a2b4732f839d2d3b2b08d511
I20260806 16:17:37.094463 23427 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:52043
I20260806 16:17:37.106042 23716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.092191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44551 (local address 127.11.245.129:48005)
0806 16:17:37.092558 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:37.092573 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:37.092711 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:37.093063 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:37.093066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:37.093347 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:37.093501 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.093508 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.094587 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.094593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.095392 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.095403 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.096096 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.096101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.096209 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.101396 (+ 5187us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:37.101411 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:37.101884 (+ 473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:37.104192 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:37.104203 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:37.104206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:37.104453 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:37.104799 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:37.104802 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:37.104804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:37.104903 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:37.105464 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:37.105467 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:37.105567 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 16:17:37.105673 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 16:17:37.105827 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":129,"threads_started":1}
I20260806 16:17:37.106145 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.092319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:48005 (local address 127.11.245.190:44551)
0806 16:17:37.092489 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:37.092494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:37.092664 (+ 170us) server_negotiation.cc:407] Connection header received
0806 16:17:37.092863 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:37.092867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:37.092930 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:37.093043 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:37.093624 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.094430 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.095521 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.095810 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.095934 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.102065 (+ 6131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:37.102069 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:37.102072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:37.102105 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:37.104013 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:37.104570 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:37.104576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:37.104578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:37.104680 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:37.105014 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:37.105018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:37.105020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:37.105328 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:37.105457 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:37.105765 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 16:17:37.105915 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":82}
WARNING: no policy specified for oryx/127.11.245.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.245.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.130.keytab.
Entry for principal oryx/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.130.keytab.
I20260806 16:17:37.128436 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:44551
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.130.keytab
--principal=oryx/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:37.295939 23719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:37.296399 23719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:37.296455 23719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:37.301388 23719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:37.301509 23719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:37.301559 23719 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:17:37.301599 23719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:17:37.308104 23719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.130.keytab
--principal=oryx/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.245.190:44551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:37.309684 23719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:37.311022 23719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:37.323601 23729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:37.323738 23719 server_base.cc:1034] running on GCE node
W20260806 16:17:37.323616 23731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:37.323601 23728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:37.324610 23719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:37.325438 23719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:37.326655 23719 hybrid_clock.cc:648] HybridClock initialized: now 1786033057326628 us; error 43 us; skew 500 ppm
Aug 06 16:17:37 dist-test-slave-t3q3 krb5kdc[23370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033057, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:37.330359 23719 init.cc:377] Logged in from keytab as oryx/127.11.245.130@KRBTEST.COM (short username oryx)
I20260806 16:17:37.331562 23719 webserver.cc:460] Webserver started at http://127.11.245.130:38369/ using document root <none> and password file <none>
I20260806 16:17:37.332485 23719 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:37.332592 23719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:37.332949 23719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:37.335680 23719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "5dc373c30b3a4497bec1315220409bc6"
format_stamp: "Formatted at 2026-08-06 16:17:37 on dist-test-slave-t3q3"
server_key: "784027745c32fc1a9e00793823118c94"
server_key_iv: "e0e4c11c00e75e741d8d026301ffe28d"
server_key_version: "encryptionkey@0"
I20260806 16:17:37.336505 23719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "5dc373c30b3a4497bec1315220409bc6"
format_stamp: "Formatted at 2026-08-06 16:17:37 on dist-test-slave-t3q3"
server_key: "784027745c32fc1a9e00793823118c94"
server_key_iv: "e0e4c11c00e75e741d8d026301ffe28d"
server_key_version: "encryptionkey@0"
I20260806 16:17:37.341516 23719 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:17:37.345151 23741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:37.347314 23719 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:17:37.347537 23719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "5dc373c30b3a4497bec1315220409bc6"
format_stamp: "Formatted at 2026-08-06 16:17:37 on dist-test-slave-t3q3"
server_key: "784027745c32fc1a9e00793823118c94"
server_key_iv: "e0e4c11c00e75e741d8d026301ffe28d"
server_key_version: "encryptionkey@0"
I20260806 16:17:37.347728 23719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:37.366056 23719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:37.370304 23719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:37.370581 23719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:37.372648 23719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:37.372728 23719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:37.372814 23719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:37.372861 23719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 16:17:37 dist-test-slave-t3q3 krb5kdc[23370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033057, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM
I20260806 16:17:37.386267 23719 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:40917
I20260806 16:17:37.386353 23892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:40917 every 8 connection(s)
I20260806 16:17:37.387656 23719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:17:37.390579 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.373755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:37653 (local address 127.11.245.190:44551)
0806 16:17:37.373949 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:37.373955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:37.375147 (+ 1192us) server_negotiation.cc:407] Connection header received
0806 16:17:37.376734 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:37.376742 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:37.376837 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:37.376959 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:37.379035 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.380029 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.381720 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.382158 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.382283 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.385979 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:37.385987 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:37.385993 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:37.386048 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:37.387867 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:37.388662 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:37.388668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:37.388672 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:37.388783 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:37.389261 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:37.389265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:37.389266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:37.389518 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:37.389656 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:37.390276 (+ 620us) server_negotiation.cc:299] Negotiation successful
0806 16:17:37.390418 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 16:17:37.391785 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.374315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44551 (local address 127.11.245.130:37653)
0806 16:17:37.374923 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:37.374988 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:37.376370 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:37.377227 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:37.377241 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:37.377829 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:37.378799 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.378822 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.380210 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.380214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.381558 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.381570 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.382348 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.382351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.382550 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.383388 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:37.383417 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:37.385686 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:37.388016 (+ 2330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:37.388024 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:37.388042 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:37.388500 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:37.388902 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:37.388909 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:37.388914 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:37.389155 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:37.389649 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:37.389656 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:37.389965 (+ 309us) client_negotiation.cc:769] Sending connection context
0806 16:17:37.390214 (+ 249us) client_negotiation.cc:240] Negotiation successful
0806 16:17:37.390597 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":175,"threads_started":1}
I20260806 16:17:37.393445 23893 heartbeater.cc:347] Connected to a master server at 127.11.245.190:44551
I20260806 16:17:37.393651 23893 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:37.394420 23893 heartbeater.cc:511] Master 127.11.245.190:44551 requested a full tablet report, sending...
I20260806 16:17:37.395599 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 23719
I20260806 16:17:37.395769 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:17:37.396025 23426 ts_manager.cc:194] Registered new tserver with Master: 5dc373c30b3a4497bec1315220409bc6 (127.11.245.130:40917)
I20260806 16:17:37.396358 12246 external_mini_cluster.cc:1428] Setting key 526a0d5e7618d630b42a5312093ba6be
I20260806 16:17:37.397102 23426 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:37653
I20260806 16:17:37.406070 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.397666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:45153 (local address 127.11.245.190:44551)
0806 16:17:37.397827 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:37.397832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:37.397847 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:17:37.397892 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:37.397897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:37.397950 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:37.398055 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:37.398626 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.399364 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.400636 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.401012 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.401142 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.402498 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:37.402502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:37.402507 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:37.402535 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:37.404000 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:37.404625 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:37.404629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:37.404631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:37.404720 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:37.405029 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:37.405032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:37.405035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:37.405295 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:37.405412 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:37.405709 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 16:17:37.405854 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 16:17:37.406090 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.396844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44551 (local address 127.11.245.130:45153)
0806 16:17:37.397293 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:37.397310 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:37.397427 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:37.398085 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:37.398088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:37.398302 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:37.398492 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.398500 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.399539 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.399543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.400512 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.400522 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.401142 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.401144 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.401223 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.401942 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:37.401958 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:37.402352 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:37.404274 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:37.404277 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:37.404280 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:37.404518 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:37.404822 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:37.404825 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:37.404827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:37.404921 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:37.405413 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:37.405417 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:37.405524 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 16:17:37.405646 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 16:17:37.405858 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":22,"thread_start_us":162,"threads_started":1}
WARNING: no policy specified for oryx/127.11.245.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.245.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.131.keytab.
Entry for principal oryx/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.131.keytab.
I20260806 16:17:37.436439 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:44551
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.131.keytab
--principal=oryx/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:37.588989 23904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:37.589340 23904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:37.589394 23904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:37.593686 23904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:37.593755 23904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:37.593798 23904 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:17:37.593833 23904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:17:37.599789 23904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.131.keytab
--principal=oryx/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.245.190:44551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:37.601336 23904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:37.602872 23904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:37.616178 23915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:37.616156 23912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:37.616219 23904 server_base.cc:1034] running on GCE node
W20260806 16:17:37.616158 23913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:37.617020 23904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:37.617856 23904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:37.619060 23904 hybrid_clock.cc:648] HybridClock initialized: now 1786033057619022 us; error 50 us; skew 500 ppm
Aug 06 16:17:37 dist-test-slave-t3q3 krb5kdc[23370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033057, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:37.622426 23904 init.cc:377] Logged in from keytab as oryx/127.11.245.131@KRBTEST.COM (short username oryx)
I20260806 16:17:37.623644 23904 webserver.cc:460] Webserver started at http://127.11.245.131:43283/ using document root <none> and password file <none>
I20260806 16:17:37.624481 23904 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:37.624590 23904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:37.624874 23904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:37.627442 23904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "c1d92c844ebd49b786d71f6904eb3554"
format_stamp: "Formatted at 2026-08-06 16:17:37 on dist-test-slave-t3q3"
server_key: "4f34f4249dc931d9e417e4eb8ee75690"
server_key_iv: "3420f3dfa53b823d620ef4b037a56057"
server_key_version: "encryptionkey@0"
I20260806 16:17:37.628230 23904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "c1d92c844ebd49b786d71f6904eb3554"
format_stamp: "Formatted at 2026-08-06 16:17:37 on dist-test-slave-t3q3"
server_key: "4f34f4249dc931d9e417e4eb8ee75690"
server_key_iv: "3420f3dfa53b823d620ef4b037a56057"
server_key_version: "encryptionkey@0"
I20260806 16:17:37.633288 23904 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:17:37.636436 23927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:37.638057 23904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 16:17:37.638231 23904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "c1d92c844ebd49b786d71f6904eb3554"
format_stamp: "Formatted at 2026-08-06 16:17:37 on dist-test-slave-t3q3"
server_key: "4f34f4249dc931d9e417e4eb8ee75690"
server_key_iv: "3420f3dfa53b823d620ef4b037a56057"
server_key_version: "encryptionkey@0"
I20260806 16:17:37.638376 23904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:37.655644 23904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:37.659534 23904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:37.659775 23904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:37.661816 23904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:37.661902 23904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:37.661984 23904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:37.662029 23904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 16:17:37 dist-test-slave-t3q3 krb5kdc[23370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033057, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM
I20260806 16:17:37.675786 23904 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:41985
I20260806 16:17:37.675858 24107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:41985 every 8 connection(s)
I20260806 16:17:37.677043 23904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:17:37.677570 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 23904
I20260806 16:17:37.677742 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:17:37.678181 12246 external_mini_cluster.cc:1428] Setting key 651ede0eb7e31bf3ce3dcec1a4cd7cba
I20260806 16:17:37.678797 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.662764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:59101 (local address 127.11.245.190:44551)
0806 16:17:37.662923 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:37.662928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:37.664317 (+ 1389us) server_negotiation.cc:407] Connection header received
0806 16:17:37.665678 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:37.665686 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:37.665780 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:37.665895 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:37.667926 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.668947 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.670652 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.670904 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.671037 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.674141 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:37.674146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:37.674150 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:37.674186 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:37.676177 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:37.676766 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:37.676771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:37.676773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:37.676873 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:37.677304 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:37.677307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:37.677309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:37.677536 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:37.677659 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:37.678432 (+ 773us) server_negotiation.cc:299] Negotiation successful
0806 16:17:37.678601 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 16:17:37.680047 23938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.663476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44551 (local address 127.11.245.131:59101)
0806 16:17:37.664144 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:37.664196 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:37.665308 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:37.666265 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:37.666277 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:37.666819 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:37.667699 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.667726 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.669112 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.669116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.670493 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.670506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.671118 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.671123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.671337 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.672054 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:37.672118 (+ 64us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:37.673919 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:37.676318 (+ 2399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:37.676324 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:37.676348 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:37.676638 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:37.676974 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:37.676979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:37.676981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:37.677174 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:37.677717 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:37.677724 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:37.678114 (+ 390us) client_negotiation.cc:769] Sending connection context
0806 16:17:37.678382 (+ 268us) client_negotiation.cc:240] Negotiation successful
0806 16:17:37.678805 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":172,"threads_started":1}
I20260806 16:17:37.681623 24108 heartbeater.cc:347] Connected to a master server at 127.11.245.190:44551
I20260806 16:17:37.681818 24108 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:37.682526 24108 heartbeater.cc:511] Master 127.11.245.190:44551 requested a full tablet report, sending...
I20260806 16:17:37.683893 23426 ts_manager.cc:194] Registered new tserver with Master: c1d92c844ebd49b786d71f6904eb3554 (127.11.245.131:41985)
I20260806 16:17:37.684878 23426 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:59101
I20260806 16:17:37.693948 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.684417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44551 (local address 127.11.245.131:49287)
0806 16:17:37.684796 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:37.684813 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:37.684955 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:37.685447 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:37.685450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:37.685760 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:37.685929 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.685936 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.686801 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.686805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.687487 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.687499 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.688043 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.688047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.688183 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.688817 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:37.688831 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:37.689404 (+ 573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:37.691845 (+ 2441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:37.691849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:37.691854 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:37.692177 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:37.692646 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:37.692649 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:37.692652 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:37.692763 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:37.693317 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:37.693320 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:37.693445 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 16:17:37.693570 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 16:17:37.693748 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":155,"threads_started":1}
I20260806 16:17:37.694020 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.684486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:49287 (local address 127.11.245.190:44551)
0806 16:17:37.684659 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:37.684664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:37.684922 (+ 258us) server_negotiation.cc:407] Connection header received
0806 16:17:37.685113 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:37.685116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:37.685229 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:37.685349 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:37.686070 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.686657 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.687655 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.687897 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.688035 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.689566 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:37.689570 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:37.689572 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:37.689599 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:37.691699 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:37.692331 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:37.692336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:37.692338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:37.692483 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:37.692895 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:37.692902 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:37.692905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:37.693210 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:37.693337 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:37.693656 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 16:17:37.693815 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 16:17:37.699388 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 16:17:37 dist-test-slave-t3q3 krb5kdc[23370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033057, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 16:17:37 dist-test-slave-t3q3 krb5kdc[23370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:37 dist-test-slave-t3q3 krb5kdc[23370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 16:17:37.732453 23500 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:37.723838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38134 (local address 127.11.245.190:44551)
0806 16:17:37.724035 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:37.724042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:37.724092 (+ 50us) server_negotiation.cc:407] Connection header received
0806 16:17:37.724325 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:37.724330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:37.724398 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:37.724494 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:37.725317 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.725917 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.726847 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.727103 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.727215 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.732216 (+ 5001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38134: BlockingRecv error: recv got EOF from 127.0.0.1:38134 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 16:17:37 dist-test-slave-t3q3 krb5kdc[23370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033057, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM
I20260806 16:17:37.744324 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.734333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38150 (local address 127.11.245.190:44551)
0806 16:17:37.734538 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:37.734544 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:37.734678 (+ 134us) server_negotiation.cc:407] Connection header received
0806 16:17:37.734877 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:37.734881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:37.734936 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:37.735044 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:37.735777 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.736434 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.737316 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.737553 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.737673 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.740558 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:37.740562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:37.740565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:37.740600 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:37.742300 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:37.742846 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:37.742850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:37.742851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:37.742985 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:37.743321 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:37.743324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:37.743326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:37.743575 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:37.743706 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:37.743943 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 16:17:37.744141 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 16:17:37.747902 23426 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:17:37.752321 23426 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:17:37.769923 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.763733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59490 (local address 127.11.245.129:36611)
0806 16:17:37.764247 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:37.764253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:37.764598 (+ 345us) server_negotiation.cc:407] Connection header received
0806 16:17:37.764844 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:37.764849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:37.765087 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:37.765213 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:37.765665 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.766487 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.768388 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.769188 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.769321 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.769445 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:37.769598 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 16:17:37.769729 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":139,"threads_started":1}
I20260806 16:17:37.770049 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.763366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57530 (local address 127.11.245.130:40917)
0806 16:17:37.764009 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:37.764023 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:37.764050 (+ 27us) server_negotiation.cc:407] Connection header received
0806 16:17:37.764178 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:37.764183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:37.764427 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:37.764773 (+ 346us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:37.765017 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.766242 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.768118 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.768995 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.769178 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.769368 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:37.769701 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 16:17:37.769849 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":150,"threads_started":1}
I20260806 16:17:37.770535 24139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.763666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:36611 (local address 127.0.0.1:59490)
0806 16:17:37.764493 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:37.764510 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:37.764681 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:37.765405 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:37.765408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:37.765427 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:37.765523 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.765528 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.766633 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.766636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.768234 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.768247 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.769337 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.769342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.769419 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.769440 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 16:17:37.769545 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 16:17:37.770336 (+ 791us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":782,"spinlock_wait_cycles":15616,"thread_start_us":101,"threads_started":1}
I20260806 16:17:37.770735 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.762802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:40917 (local address 127.0.0.1:57530)
0806 16:17:37.763620 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:37.763636 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:37.763727 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:37.764620 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:37.764628 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:37.764689 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:37.764901 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.764907 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.766381 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.766385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.767951 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.767962 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.769160 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.769164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.769338 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.769398 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 16:17:37.769552 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 16:17:37.770592 (+ 1040us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"mutex_wait_us":166,"spinlock_wait_cycles":12544,"thread_start_us":139,"threads_started":1}
I20260806 16:17:37.770891 24135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.762628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:41985 (local address 127.0.0.1:60122)
0806 16:17:37.763197 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:37.763269 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:37.763429 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:37.764620 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:37.764627 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:37.764696 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:37.764884 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.764893 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.766979 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.766984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.768234 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.768247 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.770438 (+ 2191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.770442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.770532 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.770560 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 16:17:37.770613 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 16:17:37.770703 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":144,"spinlock_wait_cycles":63744,"thread_start_us":96,"threads_started":1}
I20260806 16:17:37.771694 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.762791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60122 (local address 127.11.245.131:41985)
0806 16:17:37.763941 (+ 1150us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:37.763956 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:37.763986 (+ 30us) server_negotiation.cc:407] Connection header received
0806 16:17:37.764148 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:37.764154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:37.764417 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:37.764624 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:37.765672 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.766831 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.769451 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.770294 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.771218 (+ 924us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.771365 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:37.771417 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 16:17:37.771524 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1027,"thread_start_us":211,"threads_started":1}
I20260806 16:17:37.774143 23618 tablet_service.cc:1467] Processing CreateTablet for tablet 59124e36973b48219a80f85a51a75c29 (DEFAULT_TABLE table=test-table [id=4379874464b14ed4a53d0be8e5e56b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:37.774164 23805 tablet_service.cc:1467] Processing CreateTablet for tablet 59124e36973b48219a80f85a51a75c29 (DEFAULT_TABLE table=test-table [id=4379874464b14ed4a53d0be8e5e56b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:37.774380 24001 tablet_service.cc:1467] Processing CreateTablet for tablet 59124e36973b48219a80f85a51a75c29 (DEFAULT_TABLE table=test-table [id=4379874464b14ed4a53d0be8e5e56b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:37.775560 24001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59124e36973b48219a80f85a51a75c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:37.775556 23805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59124e36973b48219a80f85a51a75c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:37.775558 23618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59124e36973b48219a80f85a51a75c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:37.783978 24141 tablet_bootstrap.cc:492] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058: Bootstrap starting.
I20260806 16:17:37.786554 24142 tablet_bootstrap.cc:492] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554: Bootstrap starting.
I20260806 16:17:37.787304 24141 tablet_bootstrap.cc:654] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:37.788374 24141 log.cc:824] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:37.788542 24143 tablet_bootstrap.cc:492] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6: Bootstrap starting.
I20260806 16:17:37.789127 24142 tablet_bootstrap.cc:654] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:37.790164 24142 log.cc:824] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:37.791157 24141 tablet_bootstrap.cc:492] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058: No bootstrap required, opened a new log
I20260806 16:17:37.791157 24143 tablet_bootstrap.cc:654] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:37.791424 24141 ts_tablet_manager.cc:1397] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 16:17:37.792119 24143 log.cc:824] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:37.792644 24142 tablet_bootstrap.cc:492] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554: No bootstrap required, opened a new log
I20260806 16:17:37.792887 24142 ts_tablet_manager.cc:1397] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 16:17:37.795082 24143 tablet_bootstrap.cc:492] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6: No bootstrap required, opened a new log
I20260806 16:17:37.795315 24143 ts_tablet_manager.cc:1397] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 16:17:37.796949 24142 raft_consensus.cc:348] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } }
I20260806 16:17:37.797274 24142 raft_consensus.cc:374] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:37.797111 24141 raft_consensus.cc:348] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } }
I20260806 16:17:37.797345 24142 raft_consensus.cc:729] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1d92c844ebd49b786d71f6904eb3554, State: Initialized, Role: FOLLOWER
I20260806 16:17:37.797398 24141 raft_consensus.cc:374] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:37.797477 24141 raft_consensus.cc:729] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7929ac83634d43659b18bc9402685058, State: Initialized, Role: FOLLOWER
I20260806 16:17:37.797967 24142 consensus_queue.cc:260] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } }
I20260806 16:17:37.797994 24141 consensus_queue.cc:260] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } }
I20260806 16:17:37.798877 24108 heartbeater.cc:503] Master 127.11.245.190:44551 was elected leader, sending a full tablet report...
I20260806 16:17:37.799353 24141 ts_tablet_manager.cc:1428] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260806 16:17:37.799774 23707 heartbeater.cc:503] Master 127.11.245.190:44551 was elected leader, sending a full tablet report...
I20260806 16:17:37.799974 24143 raft_consensus.cc:348] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } }
I20260806 16:17:37.800321 24143 raft_consensus.cc:374] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:37.800405 24143 raft_consensus.cc:729] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc373c30b3a4497bec1315220409bc6, State: Initialized, Role: FOLLOWER
I20260806 16:17:37.800894 24142 ts_tablet_manager.cc:1428] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 16:17:37.800935 24143 consensus_queue.cc:260] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } }
I20260806 16:17:37.801733 23893 heartbeater.cc:503] Master 127.11.245.190:44551 was elected leader, sending a full tablet report...
I20260806 16:17:37.802076 24143 ts_tablet_manager.cc:1428] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260806 16:17:37.836664 23708 tablet.cc:2369] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:17:37.881212 24149 raft_consensus.cc:482] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:37.881706 24150 raft_consensus.cc:482] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:37.881644 24149 raft_consensus.cc:504] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } }
I20260806 16:17:37.881994 24150 raft_consensus.cc:504] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } }
I20260806 16:17:37.883392 24150 leader_election.cc:288] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1d92c844ebd49b786d71f6904eb3554 (127.11.245.131:41985), 7929ac83634d43659b18bc9402685058 (127.11.245.129:36611)
I20260806 16:17:37.883466 24149 leader_election.cc:288] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1d92c844ebd49b786d71f6904eb3554 (127.11.245.131:41985), 5dc373c30b3a4497bec1315220409bc6 (127.11.245.130:40917)
W20260806 16:17:37.889282 23894 tablet.cc:2369] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:17:37.890303 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.884505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:52651 (local address 127.11.245.130:40917)
0806 16:17:37.884695 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:37.884702 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:37.884720 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:17:37.884913 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:37.884916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:37.885000 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:37.885051 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:37.885476 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.886712 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.888183 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.889163 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.889391 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.889511 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:37.890050 (+ 539us) server_negotiation.cc:299] Negotiation successful
0806 16:17:37.890142 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"spinlock_wait_cycles":768}
I20260806 16:17:37.890651 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.883861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:41985 (local address 127.11.245.130:38765)
0806 16:17:37.884017 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:37.884035 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:37.884202 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:37.884897 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:37.884904 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:37.884923 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:37.885130 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.885140 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.886425 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.886429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.887798 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.887810 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.890198 (+ 2388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.890202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.890319 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.890354 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 16:17:37.890408 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 16:17:37.890492 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"spinlock_wait_cycles":1280}
I20260806 16:17:37.890722 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.884790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:35843 (local address 127.11.245.129:36611)
0806 16:17:37.884916 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:37.884921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:37.884934 (+ 13us) server_negotiation.cc:407] Connection header received
0806 16:17:37.885006 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:37.885009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:37.885058 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:37.885125 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:37.885528 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.886362 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.887964 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.889887 (+ 1923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.890000 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.890055 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:37.890416 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 16:17:37.890510 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 16:17:37.890734 24151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.883812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:40917 (local address 127.11.245.129:52651)
0806 16:17:37.884642 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:37.884657 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:37.884781 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:37.885195 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:37.885198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:37.885216 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:37.885326 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.885334 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.886841 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.886845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.888034 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.888043 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.889700 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.889705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.889866 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.889897 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 16:17:37.890487 (+ 590us) client_negotiation.cc:240] Negotiation successful
0806 16:17:37.890547 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":390,"thread_start_us":115,"threads_started":1}
I20260806 16:17:37.890909 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.883966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:58209 (local address 127.11.245.131:41985)
0806 16:17:37.884416 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:37.884423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:37.884649 (+ 226us) server_negotiation.cc:407] Connection header received
0806 16:17:37.885211 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:37.885218 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:37.885266 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:37.885313 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:37.885725 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.886451 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.888139 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.888713 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.888817 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.888886 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:37.890631 (+ 1745us) server_negotiation.cc:299] Negotiation successful
0806 16:17:37.890736 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"spinlock_wait_cycles":1408}
I20260806 16:17:37.891453 23830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59124e36973b48219a80f85a51a75c29" candidate_uuid: "7929ac83634d43659b18bc9402685058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc373c30b3a4497bec1315220409bc6" is_pre_election: true
I20260806 16:17:37.891685 24153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.884092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:38765 (local address 127.11.245.131:41985)
0806 16:17:37.884568 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:37.884580 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:37.884602 (+ 22us) server_negotiation.cc:407] Connection header received
0806 16:17:37.884683 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:37.884686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:37.884746 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:37.885238 (+ 492us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:37.885404 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.886296 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.889467 (+ 3171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.890048 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.890956 (+ 908us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.891015 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:37.891055 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 16:17:37.891135 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":408,"spinlock_wait_cycles":21120,"thread_start_us":162,"threads_started":1}
I20260806 16:17:37.891829 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.883803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:41985 (local address 127.11.245.129:58209)
0806 16:17:37.884548 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:37.884563 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:37.885066 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:37.885452 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:37.885454 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:37.885470 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:37.885591 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.885597 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.886589 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.886592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.887953 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.887966 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.888987 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.888990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.890257 (+ 1267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.890289 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 16:17:37.891546 (+ 1257us) client_negotiation.cc:240] Negotiation successful
0806 16:17:37.891646 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":95,"thread_start_us":195,"threads_started":1}
I20260806 16:17:37.891776 23830 raft_consensus.cc:2454] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7929ac83634d43659b18bc9402685058 in term 0.
I20260806 16:17:37.892256 24155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.884225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:36611 (local address 127.11.245.130:35843)
0806 16:17:37.884838 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:37.884852 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:37.884931 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:37.885241 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:37.885244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:37.885279 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:37.885405 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.885413 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.886509 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.886512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.887809 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:37.887820 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.890142 (+ 2322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.890145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:37.890254 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.890273 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 16:17:37.891041 (+ 768us) client_negotiation.cc:240] Negotiation successful
0806 16:17:37.891118 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":3,"thread_start_us":111,"threads_started":1}
I20260806 16:17:37.892385 23552 leader_election.cc:302] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dc373c30b3a4497bec1315220409bc6, 7929ac83634d43659b18bc9402685058; no voters:
I20260806 16:17:37.892462 24030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59124e36973b48219a80f85a51a75c29" candidate_uuid: "5dc373c30b3a4497bec1315220409bc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1d92c844ebd49b786d71f6904eb3554" is_pre_election: true
I20260806 16:17:37.892484 24029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59124e36973b48219a80f85a51a75c29" candidate_uuid: "7929ac83634d43659b18bc9402685058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1d92c844ebd49b786d71f6904eb3554" is_pre_election: true
I20260806 16:17:37.892735 24149 raft_consensus.cc:2781] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:37.892858 24030 raft_consensus.cc:2454] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dc373c30b3a4497bec1315220409bc6 in term 0.
I20260806 16:17:37.892874 24149 raft_consensus.cc:482] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:37.893011 24149 raft_consensus.cc:3037] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:37.893505 23642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59124e36973b48219a80f85a51a75c29" candidate_uuid: "5dc373c30b3a4497bec1315220409bc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7929ac83634d43659b18bc9402685058" is_pre_election: true
I20260806 16:17:37.893610 23742 leader_election.cc:302] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dc373c30b3a4497bec1315220409bc6, c1d92c844ebd49b786d71f6904eb3554; no voters:
I20260806 16:17:37.893937 24150 raft_consensus.cc:2781] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:37.894078 24150 raft_consensus.cc:482] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:37.894133 24150 raft_consensus.cc:3037] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:37.894788 24149 raft_consensus.cc:504] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } }
I20260806 16:17:37.895117 23642 raft_consensus.cc:2379] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5dc373c30b3a4497bec1315220409bc6 in current term 1: Already voted for candidate 7929ac83634d43659b18bc9402685058 in this term.
I20260806 16:17:37.895503 24150 raft_consensus.cc:504] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } }
I20260806 16:17:37.895592 24030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59124e36973b48219a80f85a51a75c29" candidate_uuid: "7929ac83634d43659b18bc9402685058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1d92c844ebd49b786d71f6904eb3554"
I20260806 16:17:37.895733 24030 raft_consensus.cc:3037] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:37.895779 23830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59124e36973b48219a80f85a51a75c29" candidate_uuid: "7929ac83634d43659b18bc9402685058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc373c30b3a4497bec1315220409bc6"
I20260806 16:17:37.895948 24150 leader_election.cc:288] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [CANDIDATE]: Term 1 election: Requested vote from peers c1d92c844ebd49b786d71f6904eb3554 (127.11.245.131:41985), 7929ac83634d43659b18bc9402685058 (127.11.245.129:36611)
I20260806 16:17:37.895967 23830 raft_consensus.cc:2379] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7929ac83634d43659b18bc9402685058 in current term 1: Already voted for candidate 5dc373c30b3a4497bec1315220409bc6 in this term.
I20260806 16:17:37.896504 24029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59124e36973b48219a80f85a51a75c29" candidate_uuid: "5dc373c30b3a4497bec1315220409bc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1d92c844ebd49b786d71f6904eb3554"
I20260806 16:17:37.896610 24149 leader_election.cc:288] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [CANDIDATE]: Term 1 election: Requested vote from peers c1d92c844ebd49b786d71f6904eb3554 (127.11.245.131:41985), 5dc373c30b3a4497bec1315220409bc6 (127.11.245.130:40917)
I20260806 16:17:37.896867 23642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59124e36973b48219a80f85a51a75c29" candidate_uuid: "5dc373c30b3a4497bec1315220409bc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7929ac83634d43659b18bc9402685058"
I20260806 16:17:37.897066 23642 raft_consensus.cc:2379] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5dc373c30b3a4497bec1315220409bc6 in current term 1: Already voted for candidate 7929ac83634d43659b18bc9402685058 in this term.
I20260806 16:17:37.897387 23742 leader_election.cc:302] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5dc373c30b3a4497bec1315220409bc6; no voters: 7929ac83634d43659b18bc9402685058, c1d92c844ebd49b786d71f6904eb3554
I20260806 16:17:37.897387 24030 raft_consensus.cc:2454] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7929ac83634d43659b18bc9402685058 in term 1.
I20260806 16:17:37.897637 24150 raft_consensus.cc:2726] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 16:17:37.897747 23551 leader_election.cc:302] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7929ac83634d43659b18bc9402685058, c1d92c844ebd49b786d71f6904eb3554; no voters: 5dc373c30b3a4497bec1315220409bc6
I20260806 16:17:37.897970 24149 raft_consensus.cc:2781] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:37.898250 24149 raft_consensus.cc:686] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [term 1 LEADER]: Becoming Leader. State: Replica: 7929ac83634d43659b18bc9402685058, State: Running, Role: LEADER
I20260806 16:17:37.898656 24149 consensus_queue.cc:237] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } }
I20260806 16:17:37.902863 23425 catalog_manager.cc:5495] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7929ac83634d43659b18bc9402685058 (127.11.245.129). New cstate: current_term: 1 leader_uuid: "7929ac83634d43659b18bc9402685058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } health_report { overall_health: UNKNOWN } } }
W20260806 16:17:37.928570 24109 tablet.cc:2369] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:17:37.930169 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:37.925662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59500 (local address 127.11.245.129:36611)
0806 16:17:37.925864 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:37.925869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:37.925883 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:17:37.925989 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:37.925992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:37.926063 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:37.926193 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:37.926508 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.927078 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.928027 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:37.928396 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:37.928524 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:37.928966 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:37.929664 (+ 698us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:37.929766 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:37.929875 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 16:17:37.929970 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 16:17:37.938652 24030 raft_consensus.cc:1261] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554 [term 1 FOLLOWER]: Refusing update from remote peer 7929ac83634d43659b18bc9402685058: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:37.938652 23830 raft_consensus.cc:1261] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Refusing update from remote peer 7929ac83634d43659b18bc9402685058: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:37.939911 24156 consensus_queue.cc:1035] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:37.940259 24149 consensus_queue.cc:1035] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:37.962052 23425 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:37.962344 23425 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:37.965850 23425 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27: Sending DeleteTablet for 3 replicas of tablet 59124e36973b48219a80f85a51a75c29
I20260806 16:17:37.966954 23805 tablet_service.cc:1514] Processing DeleteTablet for tablet 59124e36973b48219a80f85a51a75c29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:37 UTC) from {username='oryx', principal='oryx/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:57530
I20260806 16:17:37.966966 24001 tablet_service.cc:1514] Processing DeleteTablet for tablet 59124e36973b48219a80f85a51a75c29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:37 UTC) from {username='oryx', principal='oryx/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:60122
I20260806 16:17:37.967705 24173 tablet_replica.cc:330] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6: stopping tablet replica
I20260806 16:17:37.967717 24174 tablet_replica.cc:330] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554: stopping tablet replica
I20260806 16:17:37.967759 23425 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 16:17:37.968225 24174 raft_consensus.cc:2229] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:37.968225 24173 raft_consensus.cc:2229] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 16:17:37.968672 23425 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:17:37.968691 24174 raft_consensus.cc:2258] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:37.968691 24173 raft_consensus.cc:2258] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:37.970886 24173 ts_tablet_manager.cc:1905] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:37.970886 24174 ts_tablet_manager.cc:1905] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:37.971830 23618 tablet_service.cc:1514] Processing DeleteTablet for tablet 59124e36973b48219a80f85a51a75c29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:37 UTC) from {username='oryx', principal='oryx/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:59490
I20260806 16:17:37.974110 24175 tablet_replica.cc:330] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058: stopping tablet replica
I20260806 16:17:37.974695 24173 ts_tablet_manager.cc:1918] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:37.974853 24173 log.cc:1196] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/59124e36973b48219a80f85a51a75c29
I20260806 16:17:37.975294 24173 ts_tablet_manager.cc:1939] T 59124e36973b48219a80f85a51a75c29 P 5dc373c30b3a4497bec1315220409bc6: Deleting consensus metadata
I20260806 16:17:37.976545 24174 ts_tablet_manager.cc:1918] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:37.976699 24174 log.cc:1196] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/59124e36973b48219a80f85a51a75c29
I20260806 16:17:37.976670 23805 tablet_service.cc:1467] Processing CreateTablet for tablet 80a725b5042d46ccac1f01051d09a5c8 (DEFAULT_TABLE table=test-table [id=e8b38f6e695341a7aa05aab45fbb4cf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:37.976938 23618 tablet_service.cc:1467] Processing CreateTablet for tablet 80a725b5042d46ccac1f01051d09a5c8 (DEFAULT_TABLE table=test-table [id=e8b38f6e695341a7aa05aab45fbb4cf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:37.977052 23805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80a725b5042d46ccac1f01051d09a5c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:37.977208 24174 ts_tablet_manager.cc:1939] T 59124e36973b48219a80f85a51a75c29 P c1d92c844ebd49b786d71f6904eb3554: Deleting consensus metadata
I20260806 16:17:37.977257 23618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80a725b5042d46ccac1f01051d09a5c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:37.979874 24143 tablet_bootstrap.cc:492] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6: Bootstrap starting.
I20260806 16:17:37.980114 23407 catalog_manager.cc:4841] TS 5dc373c30b3a4497bec1315220409bc6 (127.11.245.130:40917): tablet 59124e36973b48219a80f85a51a75c29 (table test-table [id=4379874464b14ed4a53d0be8e5e56b8d]) successfully deleted
I20260806 16:17:37.980414 24001 tablet_service.cc:1467] Processing CreateTablet for tablet 80a725b5042d46ccac1f01051d09a5c8 (DEFAULT_TABLE table=test-table [id=e8b38f6e695341a7aa05aab45fbb4cf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:37.980717 24001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80a725b5042d46ccac1f01051d09a5c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:37.980985 23406 catalog_manager.cc:4841] TS c1d92c844ebd49b786d71f6904eb3554 (127.11.245.131:41985): tablet 59124e36973b48219a80f85a51a75c29 (table test-table [id=4379874464b14ed4a53d0be8e5e56b8d]) successfully deleted
I20260806 16:17:37.981150 24143 tablet_bootstrap.cc:654] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:37.981949 24175 raft_consensus.cc:2229] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:17:37.982643 24175 raft_consensus.cc:2258] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:37.982714 24143 tablet_bootstrap.cc:492] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6: No bootstrap required, opened a new log
I20260806 16:17:37.982811 24143 ts_tablet_manager.cc:1397] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:37.983455 24143 raft_consensus.cc:348] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:37.983610 24143 raft_consensus.cc:374] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:37.983656 24143 raft_consensus.cc:729] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc373c30b3a4497bec1315220409bc6, State: Initialized, Role: FOLLOWER
I20260806 16:17:37.983798 24143 consensus_queue.cc:260] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:37.983871 24141 tablet_bootstrap.cc:492] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058: Bootstrap starting.
I20260806 16:17:37.984167 24143 ts_tablet_manager.cc:1428] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:17:37.985073 24141 tablet_bootstrap.cc:654] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:37.985102 24175 ts_tablet_manager.cc:1905] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:37.986419 24142 tablet_bootstrap.cc:492] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554: Bootstrap starting.
I20260806 16:17:37.987351 24141 tablet_bootstrap.cc:492] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058: No bootstrap required, opened a new log
I20260806 16:17:37.987458 24141 ts_tablet_manager.cc:1397] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:17:37.987537 24142 tablet_bootstrap.cc:654] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:37.988126 24141 raft_consensus.cc:348] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:37.988262 24141 raft_consensus.cc:374] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:37.988303 24141 raft_consensus.cc:729] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7929ac83634d43659b18bc9402685058, State: Initialized, Role: FOLLOWER
I20260806 16:17:37.988976 24175 ts_tablet_manager.cc:1918] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:37.989043 24141 consensus_queue.cc:260] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:37.989137 24175 log.cc:1196] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/wal/wals/59124e36973b48219a80f85a51a75c29
I20260806 16:17:37.989621 24175 ts_tablet_manager.cc:1939] T 59124e36973b48219a80f85a51a75c29 P 7929ac83634d43659b18bc9402685058: Deleting consensus metadata
I20260806 16:17:37.989826 24142 tablet_bootstrap.cc:492] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554: No bootstrap required, opened a new log
I20260806 16:17:37.989938 24142 ts_tablet_manager.cc:1397] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:17:37.990197 24141 ts_tablet_manager.cc:1428] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:37.990444 24142 raft_consensus.cc:348] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:37.990572 24142 raft_consensus.cc:374] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:37.990608 24142 raft_consensus.cc:729] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1d92c844ebd49b786d71f6904eb3554, State: Initialized, Role: FOLLOWER
I20260806 16:17:37.990741 24142 consensus_queue.cc:260] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:37.991014 24142 ts_tablet_manager.cc:1428] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:17:37.992038 23406 catalog_manager.cc:4841] TS 7929ac83634d43659b18bc9402685058 (127.11.245.129:36611): tablet 59124e36973b48219a80f85a51a75c29 (table test-table [id=4379874464b14ed4a53d0be8e5e56b8d]) successfully deleted
I20260806 16:17:38.114527 24148 raft_consensus.cc:482] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:38.114748 24148 raft_consensus.cc:504] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.116705 24148 leader_election.cc:288] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7929ac83634d43659b18bc9402685058 (127.11.245.129:36611), 5dc373c30b3a4497bec1315220409bc6 (127.11.245.130:40917)
I20260806 16:17:38.122484 23938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:38.117018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:36611 (local address 127.11.245.131:36397)
0806 16:17:38.117403 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:38.117423 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:38.117549 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:38.118112 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:38.118116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:38.118141 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:38.118290 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:38.118299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.119481 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.119485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:38.120954 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:38.120977 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.122009 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.122013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:38.122129 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:38.122153 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 16:17:38.122223 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 16:17:38.122309 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":33}
I20260806 16:17:38.123028 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:38.117547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:36397 (local address 127.11.245.129:36611)
0806 16:17:38.117708 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:38.117715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:38.117733 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:17:38.117802 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:38.117805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:38.117860 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:38.117942 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:38.118459 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.119326 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.121131 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.121737 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.121866 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:38.121933 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:38.122796 (+ 863us) server_negotiation.cc:299] Negotiation successful
0806 16:17:38.122890 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 16:17:38.123246 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:38.117761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:44091 (local address 127.11.245.130:40917)
0806 16:17:38.117946 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:38.117953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:38.117969 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:17:38.118044 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:38.118049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:38.118114 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:38.118200 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:17:38.118579 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.119491 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.121250 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.121998 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.122116 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:38.122193 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:38.122937 (+ 744us) server_negotiation.cc:299] Negotiation successful
0806 16:17:38.123063 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 16:17:38.123518 23642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80a725b5042d46ccac1f01051d09a5c8" candidate_uuid: "c1d92c844ebd49b786d71f6904eb3554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7929ac83634d43659b18bc9402685058" is_pre_election: true
I20260806 16:17:38.123742 23642 raft_consensus.cc:2454] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1d92c844ebd49b786d71f6904eb3554 in term 0.
I20260806 16:17:38.124204 23928 leader_election.cc:302] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7929ac83634d43659b18bc9402685058, c1d92c844ebd49b786d71f6904eb3554; no voters:
I20260806 16:17:38.124305 24179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:38.117129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:40917 (local address 127.11.245.131:44091)
0806 16:17:38.117457 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:38.117471 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:38.117557 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:38.118224 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:38.118227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:38.118246 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:17:38.118400 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:38.118410 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.119644 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.119648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:38.121076 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:38.121091 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.122301 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.122304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:38.122431 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:38.122465 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 16:17:38.123929 (+ 1464us) client_negotiation.cc:240] Negotiation successful
0806 16:17:38.124046 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":161,"threads_started":1}
I20260806 16:17:38.124543 24148 raft_consensus.cc:2781] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:38.124629 24148 raft_consensus.cc:482] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:38.124672 24148 raft_consensus.cc:3037] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:38.125351 23830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80a725b5042d46ccac1f01051d09a5c8" candidate_uuid: "c1d92c844ebd49b786d71f6904eb3554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc373c30b3a4497bec1315220409bc6" is_pre_election: true
I20260806 16:17:38.126865 23830 raft_consensus.cc:2454] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1d92c844ebd49b786d71f6904eb3554 in term 0.
I20260806 16:17:38.128607 24148 raft_consensus.cc:504] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.129194 24148 leader_election.cc:288] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [CANDIDATE]: Term 1 election: Requested vote from peers 7929ac83634d43659b18bc9402685058 (127.11.245.129:36611), 5dc373c30b3a4497bec1315220409bc6 (127.11.245.130:40917)
I20260806 16:17:38.129698 23642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80a725b5042d46ccac1f01051d09a5c8" candidate_uuid: "c1d92c844ebd49b786d71f6904eb3554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7929ac83634d43659b18bc9402685058"
I20260806 16:17:38.129851 23642 raft_consensus.cc:3037] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:38.130086 23830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80a725b5042d46ccac1f01051d09a5c8" candidate_uuid: "c1d92c844ebd49b786d71f6904eb3554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc373c30b3a4497bec1315220409bc6"
I20260806 16:17:38.130245 23830 raft_consensus.cc:3037] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:38.131750 23642 raft_consensus.cc:2454] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1d92c844ebd49b786d71f6904eb3554 in term 1.
I20260806 16:17:38.132131 23830 raft_consensus.cc:2454] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1d92c844ebd49b786d71f6904eb3554 in term 1.
I20260806 16:17:38.132745 23930 leader_election.cc:302] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dc373c30b3a4497bec1315220409bc6, c1d92c844ebd49b786d71f6904eb3554; no voters:
I20260806 16:17:38.133188 24148 raft_consensus.cc:2781] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:38.133525 24148 raft_consensus.cc:686] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [term 1 LEADER]: Becoming Leader. State: Replica: c1d92c844ebd49b786d71f6904eb3554, State: Running, Role: LEADER
I20260806 16:17:38.134222 24148 consensus_queue.cc:237] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.137955 23427 catalog_manager.cc:5495] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1d92c844ebd49b786d71f6904eb3554 (127.11.245.131). New cstate: current_term: 1 leader_uuid: "c1d92c844ebd49b786d71f6904eb3554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } health_report { overall_health: HEALTHY } } }
Aug 06 16:17:38 dist-test-slave-t3q3 krb5kdc[23370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033056, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.190@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM
I20260806 16:17:38.173379 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:38.162601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44551 (local address 127.0.0.1:38162)
0806 16:17:38.162958 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:38.162977 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:38.163168 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:38.163473 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:38.163476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:38.164017 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:38.164220 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:38.164230 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.165061 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.165067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:38.165667 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:38.165676 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.166166 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.166170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:38.166251 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:38.167108 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:38.167121 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:38.169174 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:38.171164 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:38.171178 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:38.171189 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:38.171473 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:38.171784 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:38.171788 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:38.171790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:38.171959 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:38.172571 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:38.172583 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:38.172771 (+ 188us) client_negotiation.cc:769] Sending connection context
0806 16:17:38.172903 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 16:17:38.173130 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":150,"threads_started":1}
I20260806 16:17:38.173379 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:38.162606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38162 (local address 127.11.245.190:44551)
0806 16:17:38.162790 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:38.162798 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:38.163088 (+ 290us) server_negotiation.cc:407] Connection header received
0806 16:17:38.163298 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:38.163301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:38.163359 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:38.163447 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:38.164354 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.164942 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.165796 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.166041 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.166153 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:38.169352 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:38.169356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:38.169359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:38.169382 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:38.171011 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:38.171594 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:38.171598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:38.171600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:38.171686 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:38.172150 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:38.172155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:38.172158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:38.172454 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:38.172571 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:38.172990 (+ 419us) server_negotiation.cc:299] Negotiation successful
0806 16:17:38.173198 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":53}
I20260806 16:17:38.179896 23427 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:38162:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 16:17:38.181107 23427 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:17:38.187822 23805 tablet_service.cc:1467] Processing CreateTablet for tablet 89325673fd30456eb6790851ea06b0d0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6003c4d44a854a25af7e090ca91fee57]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:17:38.187827 23618 tablet_service.cc:1467] Processing CreateTablet for tablet 89325673fd30456eb6790851ea06b0d0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6003c4d44a854a25af7e090ca91fee57]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:17:38.188194 24001 tablet_service.cc:1467] Processing CreateTablet for tablet 89325673fd30456eb6790851ea06b0d0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6003c4d44a854a25af7e090ca91fee57]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:17:38.188453 23618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89325673fd30456eb6790851ea06b0d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:38.188452 23805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89325673fd30456eb6790851ea06b0d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:38.188643 24001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89325673fd30456eb6790851ea06b0d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:38.192340 24143 tablet_bootstrap.cc:492] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6: Bootstrap starting.
I20260806 16:17:38.192653 24142 tablet_bootstrap.cc:492] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554: Bootstrap starting.
I20260806 16:17:38.193727 24143 tablet_bootstrap.cc:654] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:38.194164 24142 tablet_bootstrap.cc:654] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:38.194600 24141 tablet_bootstrap.cc:492] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058: Bootstrap starting.
I20260806 16:17:38.195564 24142 tablet_bootstrap.cc:492] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554: No bootstrap required, opened a new log
I20260806 16:17:38.195679 24142 ts_tablet_manager.cc:1397] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:38.195698 24141 tablet_bootstrap.cc:654] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:38.196223 24142 raft_consensus.cc:348] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.196386 24142 raft_consensus.cc:374] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:38.196444 24142 raft_consensus.cc:729] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1d92c844ebd49b786d71f6904eb3554, State: Initialized, Role: FOLLOWER
I20260806 16:17:38.196614 24142 consensus_queue.cc:260] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.196874 24143 tablet_bootstrap.cc:492] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6: No bootstrap required, opened a new log
I20260806 16:17:38.196941 24142 ts_tablet_manager.cc:1428] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:17:38.196973 24143 ts_tablet_manager.cc:1397] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 16:17:38.196895 24148 tablet_replica.cc:439] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } }
I20260806 16:17:38.197057 24148 tablet_replica.cc:442] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:38.197434 24143 raft_consensus.cc:348] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.197587 24143 raft_consensus.cc:374] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:38.197633 24143 raft_consensus.cc:729] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc373c30b3a4497bec1315220409bc6, State: Initialized, Role: FOLLOWER
I20260806 16:17:38.197778 24143 consensus_queue.cc:260] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.198493 24141 tablet_bootstrap.cc:492] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058: No bootstrap required, opened a new log
I20260806 16:17:38.198601 24141 ts_tablet_manager.cc:1397] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:17:38.199115 24141 raft_consensus.cc:348] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.199237 24141 raft_consensus.cc:374] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:38.199260 24141 raft_consensus.cc:729] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7929ac83634d43659b18bc9402685058, State: Initialized, Role: FOLLOWER
I20260806 16:17:38.199424 24141 consensus_queue.cc:260] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.199733 24149 tablet_replica.cc:439] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } }
I20260806 16:17:38.199894 24149 tablet_replica.cc:442] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:38.200316 24143 ts_tablet_manager.cc:1428] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 16:17:38.200276 24150 tablet_replica.cc:439] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } }
I20260806 16:17:38.200451 24150 tablet_replica.cc:442] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:38.201160 24141 ts_tablet_manager.cc:1428] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:38.343487 24149 raft_consensus.cc:482] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:38.343739 24149 raft_consensus.cc:504] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.344400 24149 leader_election.cc:288] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dc373c30b3a4497bec1315220409bc6 (127.11.245.130:40917), c1d92c844ebd49b786d71f6904eb3554 (127.11.245.131:41985)
I20260806 16:17:38.345134 24030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89325673fd30456eb6790851ea06b0d0" candidate_uuid: "7929ac83634d43659b18bc9402685058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1d92c844ebd49b786d71f6904eb3554" is_pre_election: true
I20260806 16:17:38.345325 24030 raft_consensus.cc:2454] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7929ac83634d43659b18bc9402685058 in term 0.
I20260806 16:17:38.345705 23830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89325673fd30456eb6790851ea06b0d0" candidate_uuid: "7929ac83634d43659b18bc9402685058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc373c30b3a4497bec1315220409bc6" is_pre_election: true
I20260806 16:17:38.345901 23551 leader_election.cc:302] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7929ac83634d43659b18bc9402685058, c1d92c844ebd49b786d71f6904eb3554; no voters:
I20260806 16:17:38.345942 23830 raft_consensus.cc:2454] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7929ac83634d43659b18bc9402685058 in term 0.
I20260806 16:17:38.346164 24149 raft_consensus.cc:2781] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:38.346267 24149 raft_consensus.cc:482] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:38.346330 24149 raft_consensus.cc:3037] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:38.347832 24149 raft_consensus.cc:504] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.348357 24149 leader_election.cc:288] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [CANDIDATE]: Term 1 election: Requested vote from peers 5dc373c30b3a4497bec1315220409bc6 (127.11.245.130:40917), c1d92c844ebd49b786d71f6904eb3554 (127.11.245.131:41985)
I20260806 16:17:38.348793 23830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89325673fd30456eb6790851ea06b0d0" candidate_uuid: "7929ac83634d43659b18bc9402685058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc373c30b3a4497bec1315220409bc6"
I20260806 16:17:38.348847 24030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89325673fd30456eb6790851ea06b0d0" candidate_uuid: "7929ac83634d43659b18bc9402685058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1d92c844ebd49b786d71f6904eb3554"
I20260806 16:17:38.348933 23830 raft_consensus.cc:3037] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:38.349064 24030 raft_consensus.cc:3037] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:38.350250 23830 raft_consensus.cc:2454] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7929ac83634d43659b18bc9402685058 in term 1.
I20260806 16:17:38.350256 24030 raft_consensus.cc:2454] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7929ac83634d43659b18bc9402685058 in term 1.
I20260806 16:17:38.350648 23551 leader_election.cc:302] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7929ac83634d43659b18bc9402685058, c1d92c844ebd49b786d71f6904eb3554; no voters:
I20260806 16:17:38.350917 24149 raft_consensus.cc:2781] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:38.351034 24149 raft_consensus.cc:686] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [term 1 LEADER]: Becoming Leader. State: Replica: 7929ac83634d43659b18bc9402685058, State: Running, Role: LEADER
I20260806 16:17:38.351212 24149 consensus_queue.cc:237] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.352033 24156 tablet_replica.cc:439] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058: TxnStatusTablet state changed. Reason: New leader 7929ac83634d43659b18bc9402685058. Latest consensus state: current_term: 1 leader_uuid: "7929ac83634d43659b18bc9402685058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } }
I20260806 16:17:38.352218 24156 tablet_replica.cc:442] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:17:38.352726 24198 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 16:17:38.352942 24198 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 16:17:38.353622 23427 catalog_manager.cc:5495] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7929ac83634d43659b18bc9402685058 (127.11.245.129). New cstate: current_term: 1 leader_uuid: "7929ac83634d43659b18bc9402685058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } health_report { overall_health: UNKNOWN } } }
I20260806 16:17:38.405261 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:38.400415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:36611 (local address 127.0.0.1:59506)
0806 16:17:38.400621 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:38.400641 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:38.400771 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:38.401286 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:38.401289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:38.401298 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:17:38.401438 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:38.401444 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.402566 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.402570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:38.403623 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:38.403635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.404313 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.404317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:38.404415 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:38.404485 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:17:38.404877 (+ 392us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:17:38.404894 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 16:17:38.404963 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 16:17:38.405080 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 16:17:38.405265 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:38.400833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59506 (local address 127.11.245.129:36611)
0806 16:17:38.400983 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:38.400989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:38.401006 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:17:38.401079 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:38.401083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:38.401153 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:38.401250 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:38.401574 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.402425 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.403781 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.404186 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.404310 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:38.404650 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:38.404767 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:38.404862 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:38.405002 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 16:17:38.405081 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 16:17:38.409009 23830 raft_consensus.cc:1261] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Refusing update from remote peer 7929ac83634d43659b18bc9402685058: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:38.409046 24030 raft_consensus.cc:1261] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554 [term 1 FOLLOWER]: Refusing update from remote peer 7929ac83634d43659b18bc9402685058: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:38.409682 24149 consensus_queue.cc:1035] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:38.409894 24156 consensus_queue.cc:1035] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:38.411494 24150 tablet_replica.cc:439] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6: TxnStatusTablet state changed. Reason: New leader 7929ac83634d43659b18bc9402685058. Latest consensus state: current_term: 1 leader_uuid: "7929ac83634d43659b18bc9402685058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } }
I20260806 16:17:38.411646 24150 tablet_replica.cc:442] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:38.411592 24148 tablet_replica.cc:439] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554: TxnStatusTablet state changed. Reason: New leader 7929ac83634d43659b18bc9402685058. Latest consensus state: current_term: 1 leader_uuid: "7929ac83634d43659b18bc9402685058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } }
I20260806 16:17:38.411726 24148 tablet_replica.cc:442] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:38.413151 24149 tablet_replica.cc:439] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7929ac83634d43659b18bc9402685058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } }
I20260806 16:17:38.413303 24149 tablet_replica.cc:442] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:17:38.414650 24164 mvcc.cc:205] Tried to move back new op lower bound from 7315591407237615616 to 7315591407008194560. Current Snapshot: MvccSnapshot[applied={T|T < 7315591407237615616}]
I20260806 16:17:38.414861 24148 tablet_replica.cc:439] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7929ac83634d43659b18bc9402685058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } }
I20260806 16:17:38.414966 24148 tablet_replica.cc:442] T 89325673fd30456eb6790851ea06b0d0 P c1d92c844ebd49b786d71f6904eb3554: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:38.419266 24156 tablet_replica.cc:439] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7929ac83634d43659b18bc9402685058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } }
I20260806 16:17:38.419413 24156 tablet_replica.cc:442] T 89325673fd30456eb6790851ea06b0d0 P 7929ac83634d43659b18bc9402685058: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:17:38.420640 24150 tablet_replica.cc:439] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7929ac83634d43659b18bc9402685058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } } }
I20260806 16:17:38.420790 24150 tablet_replica.cc:442] T 89325673fd30456eb6790851ea06b0d0 P 5dc373c30b3a4497bec1315220409bc6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:17:38.422240 24161 mvcc.cc:205] Tried to move back new op lower bound from 7315591407237615616 to 7315591407008194560. Current Snapshot: MvccSnapshot[applied={T|T < 7315591407237615616 or (T in {7315591407237615616})}]
I20260806 16:17:38.425812 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:38.421201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60128 (local address 127.11.245.131:41985)
0806 16:17:38.421591 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:38.421595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:38.421608 (+ 13us) server_negotiation.cc:407] Connection header received
0806 16:17:38.421693 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:38.421695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:38.421756 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:38.421917 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:38.422232 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.422993 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.423878 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.424197 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.424318 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:38.424632 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:38.425307 (+ 675us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:38.425462 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:38.425519 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 16:17:38.425611 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":96,"threads_started":1}
I20260806 16:17:38.441059 24209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:38.436468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:36611 (local address 127.11.245.131:44557)
0806 16:17:38.436947 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:38.436963 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:38.437095 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:38.437418 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:38.437420 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:38.437428 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:17:38.437627 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:38.437642 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.438764 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.438768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:38.439494 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:38.439504 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.440175 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.440178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:38.440265 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:38.440342 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:17:38.440713 (+ 371us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:17:38.440720 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 16:17:38.440774 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 16:17:38.440862 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":192,"threads_started":1}
I20260806 16:17:38.441052 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:38.436582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:44557 (local address 127.11.245.129:36611)
0806 16:17:38.436767 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:38.436773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:38.437049 (+ 276us) server_negotiation.cc:407] Connection header received
0806 16:17:38.437213 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:38.437217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:38.437288 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:38.437387 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:38.437786 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.438628 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.439620 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.439978 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.440110 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:38.440512 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:38.440624 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:38.440700 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:38.440821 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 16:17:38.440889 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 16:17:38.451362 23830 raft_consensus.cc:1261] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Refusing update from remote peer c1d92c844ebd49b786d71f6904eb3554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:38.451699 23642 raft_consensus.cc:1261] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Refusing update from remote peer c1d92c844ebd49b786d71f6904eb3554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:38.452021 24180 consensus_queue.cc:1035] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:38.452260 24148 consensus_queue.cc:1035] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:38.463625 24160 mvcc.cc:205] Tried to move back new op lower bound from 7315591407411302400 to 7315591406122274816. Current Snapshot: MvccSnapshot[applied={T|T < 7315591407411302400}]
I20260806 16:17:38.480952 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:38.476141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:54059 (local address 127.11.245.131:41985)
0806 16:17:38.476585 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:38.476594 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:38.476937 (+ 343us) server_negotiation.cc:407] Connection header received
0806 16:17:38.477125 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:38.477129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:38.477200 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:38.477288 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:38.477586 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.478306 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.479562 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.479810 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.479937 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:38.480376 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:38.480484 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:38.480577 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:38.480674 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 16:17:38.480757 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 16:17:38.480952 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:38.476128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:41985 (local address 127.11.245.129:54059)
0806 16:17:38.476832 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:38.476848 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:38.477000 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:38.477304 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:38.477307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:38.477315 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:17:38.477456 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:38.477463 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.478482 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.478487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:38.479399 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:38.479411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.479950 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.479953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:38.480116 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:38.480208 (+ 92us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:17:38.480582 (+ 374us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:17:38.480592 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 16:17:38.480663 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 16:17:38.480774 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":181,"threads_started":1}
I20260806 16:17:38.515393 23427 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:38.515658 23427 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:38.518751 23427 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27: Sending DeleteTablet for 3 replicas of tablet 80a725b5042d46ccac1f01051d09a5c8
I20260806 16:17:38.519711 24001 tablet_service.cc:1514] Processing DeleteTablet for tablet 80a725b5042d46ccac1f01051d09a5c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:38 UTC) from {username='oryx', principal='oryx/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:60122
I20260806 16:17:38.520123 23805 tablet_service.cc:1514] Processing DeleteTablet for tablet 80a725b5042d46ccac1f01051d09a5c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:38 UTC) from {username='oryx', principal='oryx/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:57530
I20260806 16:17:38.520200 24222 tablet_replica.cc:330] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554: stopping tablet replica
I20260806 16:17:38.520530 24222 raft_consensus.cc:2229] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:17:38.521052 24222 raft_consensus.cc:2258] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:38.522046 24222 ts_tablet_manager.cc:1905] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:38.526357 23618 tablet_service.cc:1514] Processing DeleteTablet for tablet 80a725b5042d46ccac1f01051d09a5c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:38 UTC) from {username='oryx', principal='oryx/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:59490
I20260806 16:17:38.527311 24222 ts_tablet_manager.cc:1918] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:17:38.527469 24222 log.cc:1196] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/80a725b5042d46ccac1f01051d09a5c8
I20260806 16:17:38.527973 24222 ts_tablet_manager.cc:1939] T 80a725b5042d46ccac1f01051d09a5c8 P c1d92c844ebd49b786d71f6904eb3554: Deleting consensus metadata
I20260806 16:17:38.529285 23406 catalog_manager.cc:4841] TS c1d92c844ebd49b786d71f6904eb3554 (127.11.245.131:41985): tablet 80a725b5042d46ccac1f01051d09a5c8 (table test-table [id=e8b38f6e695341a7aa05aab45fbb4cf4]) successfully deleted
I20260806 16:17:38.529789 24227 tablet_replica.cc:330] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058: stopping tablet replica
I20260806 16:17:38.529904 24225 tablet_replica.cc:330] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6: stopping tablet replica
I20260806 16:17:38.530108 24225 raft_consensus.cc:2229] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:38.530135 24227 raft_consensus.cc:2229] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:38.530293 24225 raft_consensus.cc:2258] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:38.530407 24227 raft_consensus.cc:2258] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:38.531282 24227 ts_tablet_manager.cc:1905] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:38.531250 24225 ts_tablet_manager.cc:1905] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:38.535238 24227 ts_tablet_manager.cc:1918] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:17:38.535391 24227 log.cc:1196] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/wal/wals/80a725b5042d46ccac1f01051d09a5c8
I20260806 16:17:38.535734 24225 ts_tablet_manager.cc:1918] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:17:38.535760 24227 ts_tablet_manager.cc:1939] T 80a725b5042d46ccac1f01051d09a5c8 P 7929ac83634d43659b18bc9402685058: Deleting consensus metadata
I20260806 16:17:38.535830 24225 log.cc:1196] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/80a725b5042d46ccac1f01051d09a5c8
I20260806 16:17:38.536127 24225 ts_tablet_manager.cc:1939] T 80a725b5042d46ccac1f01051d09a5c8 P 5dc373c30b3a4497bec1315220409bc6: Deleting consensus metadata
I20260806 16:17:38.536985 23406 catalog_manager.cc:4841] TS 7929ac83634d43659b18bc9402685058 (127.11.245.129:36611): tablet 80a725b5042d46ccac1f01051d09a5c8 (table test-table [id=e8b38f6e695341a7aa05aab45fbb4cf4]) successfully deleted
I20260806 16:17:38.540216 23407 catalog_manager.cc:4841] TS 5dc373c30b3a4497bec1315220409bc6 (127.11.245.130:40917): tablet 80a725b5042d46ccac1f01051d09a5c8 (table test-table [id=e8b38f6e695341a7aa05aab45fbb4cf4]) successfully deleted
Aug 06 16:17:38 dist-test-slave-t3q3 krb5kdc[23370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 16:17:38 dist-test-slave-t3q3 krb5kdc[23370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:38 dist-test-slave-t3q3 krb5kdc[23370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 16:17:38.564841 23500 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:38.556889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38166 (local address 127.11.245.190:44551)
0806 16:17:38.557089 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:38.557095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:38.557357 (+ 262us) server_negotiation.cc:407] Connection header received
0806 16:17:38.557522 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:38.557526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:38.557596 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:38.557702 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:38.558553 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.559506 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.560549 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.560928 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.561055 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:38.564665 (+ 3610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38166: BlockingRecv error: recv got EOF from 127.0.0.1:38166 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 16:17:38 dist-test-slave-t3q3 krb5kdc[23370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM
I20260806 16:17:38.577299 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:38.567257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38182 (local address 127.11.245.190:44551)
0806 16:17:38.567412 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:38.567417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:38.567561 (+ 144us) server_negotiation.cc:407] Connection header received
0806 16:17:38.567723 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:38.567726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:38.567781 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:38.567848 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:38.568735 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.569310 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.570469 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.570694 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.570775 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:38.573756 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:38.573760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:38.573762 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:38.573790 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:38.575347 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:38.575815 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:38.575819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:38.575821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:38.575913 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:38.576283 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:38.576288 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:38.576290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:38.576506 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:38.576663 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:38.576934 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 16:17:38.577089 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 16:17:38.580442 23427 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38182:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:17:38.581053 23427 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:17:38.587090 24001 tablet_service.cc:1467] Processing CreateTablet for tablet 70a345f778044a3fb8b2623edea19f4f (DEFAULT_TABLE table=test-table [id=d6c24a1323ea476fb3b0f5dfdf723385]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:38.587132 23805 tablet_service.cc:1467] Processing CreateTablet for tablet 70a345f778044a3fb8b2623edea19f4f (DEFAULT_TABLE table=test-table [id=d6c24a1323ea476fb3b0f5dfdf723385]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:38.587175 23618 tablet_service.cc:1467] Processing CreateTablet for tablet 70a345f778044a3fb8b2623edea19f4f (DEFAULT_TABLE table=test-table [id=d6c24a1323ea476fb3b0f5dfdf723385]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:38.587509 23805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70a345f778044a3fb8b2623edea19f4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:38.587522 23618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70a345f778044a3fb8b2623edea19f4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:38.587522 24001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70a345f778044a3fb8b2623edea19f4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:38.590453 24142 tablet_bootstrap.cc:492] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554: Bootstrap starting.
I20260806 16:17:38.590932 24141 tablet_bootstrap.cc:492] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058: Bootstrap starting.
I20260806 16:17:38.590991 24143 tablet_bootstrap.cc:492] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6: Bootstrap starting.
I20260806 16:17:38.591662 24142 tablet_bootstrap.cc:654] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:38.591866 24141 tablet_bootstrap.cc:654] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:38.592034 24143 tablet_bootstrap.cc:654] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:38.593834 24142 tablet_bootstrap.cc:492] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554: No bootstrap required, opened a new log
I20260806 16:17:38.593936 24142 ts_tablet_manager.cc:1397] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 16:17:38.594223 24141 tablet_bootstrap.cc:492] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058: No bootstrap required, opened a new log
I20260806 16:17:38.594305 24141 ts_tablet_manager.cc:1397] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:38.594587 24143 tablet_bootstrap.cc:492] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6: No bootstrap required, opened a new log
I20260806 16:17:38.594543 24142 raft_consensus.cc:348] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.594668 24143 ts_tablet_manager.cc:1397] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:17:38.594681 24142 raft_consensus.cc:374] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:38.594717 24142 raft_consensus.cc:729] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1d92c844ebd49b786d71f6904eb3554, State: Initialized, Role: FOLLOWER
I20260806 16:17:38.594723 24141 raft_consensus.cc:348] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.594848 24141 raft_consensus.cc:374] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:38.594880 24141 raft_consensus.cc:729] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7929ac83634d43659b18bc9402685058, State: Initialized, Role: FOLLOWER
I20260806 16:17:38.594866 24142 consensus_queue.cc:260] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.595026 24141 consensus_queue.cc:260] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.595278 24142 ts_tablet_manager.cc:1428] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260806 16:17:38.595211 24143 raft_consensus.cc:348] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.595316 24141 ts_tablet_manager.cc:1428] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:17:38.595335 24143 raft_consensus.cc:374] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:38.595368 24143 raft_consensus.cc:729] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc373c30b3a4497bec1315220409bc6, State: Initialized, Role: FOLLOWER
I20260806 16:17:38.595521 24143 consensus_queue.cc:260] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.595829 24143 ts_tablet_manager.cc:1428] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:17:38.711143 24214 raft_consensus.cc:482] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:38.711349 24214 raft_consensus.cc:504] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.712007 24214 leader_election.cc:288] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dc373c30b3a4497bec1315220409bc6 (127.11.245.130:40917), c1d92c844ebd49b786d71f6904eb3554 (127.11.245.131:41985)
I20260806 16:17:38.712641 24030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70a345f778044a3fb8b2623edea19f4f" candidate_uuid: "7929ac83634d43659b18bc9402685058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1d92c844ebd49b786d71f6904eb3554" is_pre_election: true
I20260806 16:17:38.712643 23830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70a345f778044a3fb8b2623edea19f4f" candidate_uuid: "7929ac83634d43659b18bc9402685058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc373c30b3a4497bec1315220409bc6" is_pre_election: true
I20260806 16:17:38.712836 24030 raft_consensus.cc:2454] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7929ac83634d43659b18bc9402685058 in term 0.
I20260806 16:17:38.712886 23830 raft_consensus.cc:2454] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7929ac83634d43659b18bc9402685058 in term 0.
I20260806 16:17:38.713222 23551 leader_election.cc:302] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7929ac83634d43659b18bc9402685058, c1d92c844ebd49b786d71f6904eb3554; no voters:
I20260806 16:17:38.713532 24214 raft_consensus.cc:2781] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:38.713624 24214 raft_consensus.cc:482] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:38.713708 24214 raft_consensus.cc:3037] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:38.715154 24214 raft_consensus.cc:504] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.715633 24214 leader_election.cc:288] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [CANDIDATE]: Term 1 election: Requested vote from peers 5dc373c30b3a4497bec1315220409bc6 (127.11.245.130:40917), c1d92c844ebd49b786d71f6904eb3554 (127.11.245.131:41985)
I20260806 16:17:38.716147 23830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70a345f778044a3fb8b2623edea19f4f" candidate_uuid: "7929ac83634d43659b18bc9402685058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc373c30b3a4497bec1315220409bc6"
I20260806 16:17:38.716171 24030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70a345f778044a3fb8b2623edea19f4f" candidate_uuid: "7929ac83634d43659b18bc9402685058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1d92c844ebd49b786d71f6904eb3554"
I20260806 16:17:38.716329 23830 raft_consensus.cc:3037] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:38.716341 24030 raft_consensus.cc:3037] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:38.717525 24030 raft_consensus.cc:2454] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7929ac83634d43659b18bc9402685058 in term 1.
I20260806 16:17:38.717535 23830 raft_consensus.cc:2454] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7929ac83634d43659b18bc9402685058 in term 1.
I20260806 16:17:38.717959 23552 leader_election.cc:302] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dc373c30b3a4497bec1315220409bc6, 7929ac83634d43659b18bc9402685058; no voters:
I20260806 16:17:38.718185 24214 raft_consensus.cc:2781] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:38.718300 24214 raft_consensus.cc:686] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [term 1 LEADER]: Becoming Leader. State: Replica: 7929ac83634d43659b18bc9402685058, State: Running, Role: LEADER
I20260806 16:17:38.718483 24214 consensus_queue.cc:237] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.720676 23425 catalog_manager.cc:5495] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7929ac83634d43659b18bc9402685058 (127.11.245.129). New cstate: current_term: 1 leader_uuid: "7929ac83634d43659b18bc9402685058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } health_report { overall_health: UNKNOWN } } }
I20260806 16:17:38.760021 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:38.756669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59512 (local address 127.11.245.129:36611)
0806 16:17:38.756845 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:38.756849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:38.756864 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:17:38.756970 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:38.756972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:38.757026 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:38.757093 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:38.757383 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.757929 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.758786 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:38.759063 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:38.759150 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:38.759447 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:38.759540 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:38.759641 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:38.759777 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 16:17:38.759838 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 16:17:38.762763 24030 raft_consensus.cc:1261] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554 [term 1 FOLLOWER]: Refusing update from remote peer 7929ac83634d43659b18bc9402685058: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:38.762763 23830 raft_consensus.cc:1261] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Refusing update from remote peer 7929ac83634d43659b18bc9402685058: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:38.763309 24214 consensus_queue.cc:1035] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:38.763528 24149 consensus_queue.cc:1035] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:38.773120 23425 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:38.773324 23425 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:38.775790 23425 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27: Sending DeleteTablet for 3 replicas of tablet 70a345f778044a3fb8b2623edea19f4f
I20260806 16:17:38.776821 23805 tablet_service.cc:1514] Processing DeleteTablet for tablet 70a345f778044a3fb8b2623edea19f4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:38 UTC) from {username='oryx', principal='oryx/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:57530
I20260806 16:17:38.777133 24225 tablet_replica.cc:330] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6: stopping tablet replica
I20260806 16:17:38.777236 23618 tablet_service.cc:1514] Processing DeleteTablet for tablet 70a345f778044a3fb8b2623edea19f4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:38 UTC) from {username='oryx', principal='oryx/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:59490
I20260806 16:17:38.777290 24225 raft_consensus.cc:2229] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:38.777470 24225 raft_consensus.cc:2258] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:38.777499 24227 tablet_replica.cc:330] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058: stopping tablet replica
I20260806 16:17:38.777547 24001 tablet_service.cc:1514] Processing DeleteTablet for tablet 70a345f778044a3fb8b2623edea19f4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:38 UTC) from {username='oryx', principal='oryx/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:60122
I20260806 16:17:38.777664 24227 raft_consensus.cc:2229] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:17:38.777781 24222 tablet_replica.cc:330] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554: stopping tablet replica
I20260806 16:17:38.778136 24222 raft_consensus.cc:2229] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:38.778180 24227 raft_consensus.cc:2258] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:38.778443 24222 raft_consensus.cc:2258] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:38.778538 24225 ts_tablet_manager.cc:1905] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:38.778718 23425 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38182:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 16:17:38.779382 24227 ts_tablet_manager.cc:1905] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:38.779564 24222 ts_tablet_manager.cc:1905] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 16:17:38.779575 23425 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:17:38.781876 24225 ts_tablet_manager.cc:1918] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:38.781975 24225 log.cc:1196] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/70a345f778044a3fb8b2623edea19f4f
I20260806 16:17:38.782285 24225 ts_tablet_manager.cc:1939] T 70a345f778044a3fb8b2623edea19f4f P 5dc373c30b3a4497bec1315220409bc6: Deleting consensus metadata
I20260806 16:17:38.782572 24227 ts_tablet_manager.cc:1918] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:38.782675 24227 log.cc:1196] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-0/wal/wals/70a345f778044a3fb8b2623edea19f4f
I20260806 16:17:38.782745 24222 ts_tablet_manager.cc:1918] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:17:38.782853 24222 log.cc:1196] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/70a345f778044a3fb8b2623edea19f4f
I20260806 16:17:38.782969 24227 ts_tablet_manager.cc:1939] T 70a345f778044a3fb8b2623edea19f4f P 7929ac83634d43659b18bc9402685058: Deleting consensus metadata
I20260806 16:17:38.783185 24222 ts_tablet_manager.cc:1939] T 70a345f778044a3fb8b2623edea19f4f P c1d92c844ebd49b786d71f6904eb3554: Deleting consensus metadata
I20260806 16:17:38.783236 23407 catalog_manager.cc:4841] TS 5dc373c30b3a4497bec1315220409bc6 (127.11.245.130:40917): tablet 70a345f778044a3fb8b2623edea19f4f (table test-table [id=d6c24a1323ea476fb3b0f5dfdf723385]) successfully deleted
I20260806 16:17:38.784475 23406 catalog_manager.cc:4841] TS c1d92c844ebd49b786d71f6904eb3554 (127.11.245.131:41985): tablet 70a345f778044a3fb8b2623edea19f4f (table test-table [id=d6c24a1323ea476fb3b0f5dfdf723385]) successfully deleted
I20260806 16:17:38.784648 23406 catalog_manager.cc:4841] TS 7929ac83634d43659b18bc9402685058 (127.11.245.129:36611): tablet 70a345f778044a3fb8b2623edea19f4f (table test-table [id=d6c24a1323ea476fb3b0f5dfdf723385]) successfully deleted
I20260806 16:17:38.786600 23618 tablet_service.cc:1467] Processing CreateTablet for tablet 88c36cb1ad794dd08666444f16edce8d (DEFAULT_TABLE table=test-table [id=75b75d1e37e547c69fe1858f2033f25f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:38.786625 23805 tablet_service.cc:1467] Processing CreateTablet for tablet 88c36cb1ad794dd08666444f16edce8d (DEFAULT_TABLE table=test-table [id=75b75d1e37e547c69fe1858f2033f25f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:38.786755 24001 tablet_service.cc:1467] Processing CreateTablet for tablet 88c36cb1ad794dd08666444f16edce8d (DEFAULT_TABLE table=test-table [id=75b75d1e37e547c69fe1858f2033f25f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:17:38.786962 23805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88c36cb1ad794dd08666444f16edce8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:38.786957 23618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88c36cb1ad794dd08666444f16edce8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:38.787045 24001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88c36cb1ad794dd08666444f16edce8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:38.789764 24142 tablet_bootstrap.cc:492] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554: Bootstrap starting.
I20260806 16:17:38.789827 24143 tablet_bootstrap.cc:492] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6: Bootstrap starting.
I20260806 16:17:38.790843 24142 tablet_bootstrap.cc:654] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:38.790994 24143 tablet_bootstrap.cc:654] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:38.791690 24141 tablet_bootstrap.cc:492] T 88c36cb1ad794dd08666444f16edce8d P 7929ac83634d43659b18bc9402685058: Bootstrap starting.
I20260806 16:17:38.792281 24142 tablet_bootstrap.cc:492] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554: No bootstrap required, opened a new log
I20260806 16:17:38.792371 24142 ts_tablet_manager.cc:1397] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:38.792780 24141 tablet_bootstrap.cc:654] T 88c36cb1ad794dd08666444f16edce8d P 7929ac83634d43659b18bc9402685058: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:38.792876 24142 raft_consensus.cc:348] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.793036 24142 raft_consensus.cc:374] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:38.793081 24142 raft_consensus.cc:729] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1d92c844ebd49b786d71f6904eb3554, State: Initialized, Role: FOLLOWER
I20260806 16:17:38.793236 24142 consensus_queue.cc:260] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.793582 24142 ts_tablet_manager.cc:1428] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 16:17:38.793792 24143 tablet_bootstrap.cc:492] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6: No bootstrap required, opened a new log
I20260806 16:17:38.793884 24143 ts_tablet_manager.cc:1397] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:17:38.794342 24143 raft_consensus.cc:348] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.794521 24143 raft_consensus.cc:374] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:38.794590 24143 raft_consensus.cc:729] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc373c30b3a4497bec1315220409bc6, State: Initialized, Role: FOLLOWER
I20260806 16:17:38.794739 24143 consensus_queue.cc:260] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.795042 24143 ts_tablet_manager.cc:1428] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 16:17:38.797137 24141 tablet_bootstrap.cc:492] T 88c36cb1ad794dd08666444f16edce8d P 7929ac83634d43659b18bc9402685058: No bootstrap required, opened a new log
I20260806 16:17:38.797286 24141 ts_tablet_manager.cc:1397] T 88c36cb1ad794dd08666444f16edce8d P 7929ac83634d43659b18bc9402685058: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 16:17:38.797775 24141 raft_consensus.cc:348] T 88c36cb1ad794dd08666444f16edce8d P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.797930 24141 raft_consensus.cc:374] T 88c36cb1ad794dd08666444f16edce8d P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:38.797969 24141 raft_consensus.cc:729] T 88c36cb1ad794dd08666444f16edce8d P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7929ac83634d43659b18bc9402685058, State: Initialized, Role: FOLLOWER
I20260806 16:17:38.798128 24141 consensus_queue.cc:260] T 88c36cb1ad794dd08666444f16edce8d P 7929ac83634d43659b18bc9402685058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:38.798506 24141 ts_tablet_manager.cc:1428] T 88c36cb1ad794dd08666444f16edce8d P 7929ac83634d43659b18bc9402685058: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 16:17:39.068517 24180 raft_consensus.cc:482] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:17:39.068799 24180 raft_consensus.cc:504] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:39.069485 24180 leader_election.cc:288] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7929ac83634d43659b18bc9402685058 (127.11.245.129:36611), 5dc373c30b3a4497bec1315220409bc6 (127.11.245.130:40917)
I20260806 16:17:39.070207 23642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88c36cb1ad794dd08666444f16edce8d" candidate_uuid: "c1d92c844ebd49b786d71f6904eb3554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7929ac83634d43659b18bc9402685058" is_pre_election: true
I20260806 16:17:39.070210 23830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88c36cb1ad794dd08666444f16edce8d" candidate_uuid: "c1d92c844ebd49b786d71f6904eb3554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc373c30b3a4497bec1315220409bc6" is_pre_election: true
I20260806 16:17:39.070431 23642 raft_consensus.cc:2454] T 88c36cb1ad794dd08666444f16edce8d P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1d92c844ebd49b786d71f6904eb3554 in term 0.
I20260806 16:17:39.070444 23830 raft_consensus.cc:2454] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1d92c844ebd49b786d71f6904eb3554 in term 0.
I20260806 16:17:39.070940 23928 leader_election.cc:302] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7929ac83634d43659b18bc9402685058, c1d92c844ebd49b786d71f6904eb3554; no voters:
I20260806 16:17:39.071194 24180 raft_consensus.cc:2781] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:17:39.071282 24180 raft_consensus.cc:482] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:17:39.071353 24180 raft_consensus.cc:3037] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:39.072909 24180 raft_consensus.cc:504] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:39.073398 24180 leader_election.cc:288] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [CANDIDATE]: Term 1 election: Requested vote from peers 7929ac83634d43659b18bc9402685058 (127.11.245.129:36611), 5dc373c30b3a4497bec1315220409bc6 (127.11.245.130:40917)
I20260806 16:17:39.074065 23830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88c36cb1ad794dd08666444f16edce8d" candidate_uuid: "c1d92c844ebd49b786d71f6904eb3554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc373c30b3a4497bec1315220409bc6"
I20260806 16:17:39.074064 23642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88c36cb1ad794dd08666444f16edce8d" candidate_uuid: "c1d92c844ebd49b786d71f6904eb3554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7929ac83634d43659b18bc9402685058"
I20260806 16:17:39.074255 23642 raft_consensus.cc:3037] T 88c36cb1ad794dd08666444f16edce8d P 7929ac83634d43659b18bc9402685058 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:39.074255 23830 raft_consensus.cc:3037] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:39.075762 23642 raft_consensus.cc:2454] T 88c36cb1ad794dd08666444f16edce8d P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1d92c844ebd49b786d71f6904eb3554 in term 1.
I20260806 16:17:39.075765 23830 raft_consensus.cc:2454] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1d92c844ebd49b786d71f6904eb3554 in term 1.
I20260806 16:17:39.076192 23928 leader_election.cc:302] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7929ac83634d43659b18bc9402685058, c1d92c844ebd49b786d71f6904eb3554; no voters:
I20260806 16:17:39.076481 24180 raft_consensus.cc:2781] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:39.076717 24180 raft_consensus.cc:686] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [term 1 LEADER]: Becoming Leader. State: Replica: c1d92c844ebd49b786d71f6904eb3554, State: Running, Role: LEADER
I20260806 16:17:39.076913 24180 consensus_queue.cc:237] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } }
I20260806 16:17:39.079067 23427 catalog_manager.cc:5495] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1d92c844ebd49b786d71f6904eb3554 (127.11.245.131). New cstate: current_term: 1 leader_uuid: "c1d92c844ebd49b786d71f6904eb3554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1d92c844ebd49b786d71f6904eb3554" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41985 } health_report { overall_health: HEALTHY } } }
I20260806 16:17:39.111536 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:39.107817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60136 (local address 127.11.245.131:41985)
0806 16:17:39.108292 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:39.108297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:39.108308 (+ 11us) server_negotiation.cc:407] Connection header received
0806 16:17:39.108357 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:39.108359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:39.108410 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:39.108546 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:17:39.108836 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:39.109416 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:39.110314 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:39.110571 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:39.110688 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:39.110966 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:17:39.111061 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:17:39.111148 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:39.111275 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 16:17:39.111341 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":139,"threads_started":1}
I20260806 16:17:39.119480 23830 raft_consensus.cc:1261] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Refusing update from remote peer c1d92c844ebd49b786d71f6904eb3554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:39.119597 23642 raft_consensus.cc:1261] T 88c36cb1ad794dd08666444f16edce8d P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Refusing update from remote peer c1d92c844ebd49b786d71f6904eb3554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:17:39.119975 24180 consensus_queue.cc:1035] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dc373c30b3a4497bec1315220409bc6" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:39.120201 24259 consensus_queue.cc:1035] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7929ac83634d43659b18bc9402685058" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 36611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:17:39.157964 23426 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:39.158179 23426 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:17:39.161849 23426 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba27428000ad492dbb04f5985577ab27: Sending DeleteTablet for 3 replicas of tablet 88c36cb1ad794dd08666444f16edce8d
I20260806 16:17:39.163076 24001 tablet_service.cc:1514] Processing DeleteTablet for tablet 88c36cb1ad794dd08666444f16edce8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:39 UTC) from {username='oryx', principal='oryx/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:60122
I20260806 16:17:39.163117 23805 tablet_service.cc:1514] Processing DeleteTablet for tablet 88c36cb1ad794dd08666444f16edce8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:39 UTC) from {username='oryx', principal='oryx/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:57530
I20260806 16:17:39.163358 24222 tablet_replica.cc:330] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554: stopping tablet replica
I20260806 16:17:39.163367 23618 tablet_service.cc:1514] Processing DeleteTablet for tablet 88c36cb1ad794dd08666444f16edce8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:17:39 UTC) from {username='oryx', principal='oryx/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:59490
I20260806 16:17:39.163545 24222 raft_consensus.cc:2229] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:17:39.163590 24227 tablet_replica.cc:330] T 88c36cb1ad794dd08666444f16edce8d P 7929ac83634d43659b18bc9402685058: stopping tablet replica
I20260806 16:17:39.163725 24227 raft_consensus.cc:2229] T 88c36cb1ad794dd08666444f16edce8d P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:39.163838 24222 raft_consensus.cc:2258] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:39.163856 24227 raft_consensus.cc:2258] T 88c36cb1ad794dd08666444f16edce8d P 7929ac83634d43659b18bc9402685058 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:39.164024 24225 tablet_replica.cc:330] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6: stopping tablet replica
I20260806 16:17:39.164249 24225 raft_consensus.cc:2229] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:17:39.164431 24225 raft_consensus.cc:2258] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:17:39.164431 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 23530
I20260806 16:17:39.177490 24222 ts_tablet_manager.cc:1905] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:39.177613 24225 ts_tablet_manager.cc:1905] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:17:39.180706 24222 ts_tablet_manager.cc:1918] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:17:39.180861 24222 log.cc:1196] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/88c36cb1ad794dd08666444f16edce8d
I20260806 16:17:39.180820 24225 ts_tablet_manager.cc:1918] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:17:39.180943 24225 log.cc:1196] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/88c36cb1ad794dd08666444f16edce8d
I20260806 16:17:39.181195 24222 ts_tablet_manager.cc:1939] T 88c36cb1ad794dd08666444f16edce8d P c1d92c844ebd49b786d71f6904eb3554: Deleting consensus metadata
I20260806 16:17:39.181483 24225 ts_tablet_manager.cc:1939] T 88c36cb1ad794dd08666444f16edce8d P 5dc373c30b3a4497bec1315220409bc6: Deleting consensus metadata
I20260806 16:17:39.183534 23407 catalog_manager.cc:4841] TS 5dc373c30b3a4497bec1315220409bc6 (127.11.245.130:40917): tablet 88c36cb1ad794dd08666444f16edce8d (table test-table [id=75b75d1e37e547c69fe1858f2033f25f]) successfully deleted
I20260806 16:17:39.183534 23406 catalog_manager.cc:4841] TS c1d92c844ebd49b786d71f6904eb3554 (127.11.245.131:41985): tablet 88c36cb1ad794dd08666444f16edce8d (table test-table [id=75b75d1e37e547c69fe1858f2033f25f]) successfully deleted
W20260806 16:17:39.195659 23406 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:36611 (error 108)
I20260806 16:17:39.196192 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 23719
I20260806 16:17:39.204350 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:39.202195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:36611 (local address 127.0.0.1:59526)
0806 16:17:39.204010 (+ 1815us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:39.204168 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:36611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1673,"thread_start_us":961,"threads_started":1}
W20260806 16:17:39.205803 23406 catalog_manager.cc:4584] TS 7929ac83634d43659b18bc9402685058 (127.11.245.129:36611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 88c36cb1ad794dd08666444f16edce8d: Network error: Client connection negotiation failed: client connection to 127.11.245.129:36611: connect: Connection refused (error 111)
I20260806 16:17:39.216205 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 23904
I20260806 16:17:39.233996 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 23386
2026-08-06T16:17:39Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (4228 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:17:39 dist-test-slave-t3q3 krb5kdc[24276](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:17:39 dist-test-slave-t3q3 krb5kdc[24276](info): set up 2 sockets
Aug 06 16:17:39 dist-test-slave-t3q3 krb5kdc[24276](info): commencing operation
krb5kdc: starting...
W20260806 16:17:40.535763 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:17:40 dist-test-slave-t3q3 krb5kdc[24276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:17:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:17:40.700484 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:46775
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:46775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:40.857263 24292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:40.857731 24292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:40.857820 24292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:40.862562 24292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:40.862690 24292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:40.862728 24292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:40.862761 24292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:40.862821 24292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:17:40.869371 24292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:46775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:46775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:40.871093 24292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:40.872496 24292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:40.883244 24298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:40.883253 24302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:40.883383 24297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:40.883694 24292 server_base.cc:1034] running on GCE node
I20260806 16:17:40.884475 24292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:40.886050 24292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:40.887264 24292 hybrid_clock.cc:648] HybridClock initialized: now 1786033060887250 us; error 38 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T16:17:40Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1656 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:17:40 dist-test-slave-t3q3 krb5kdc[24310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:17:40 dist-test-slave-t3q3 krb5kdc[24310](info): set up 2 sockets
Aug 06 16:17:40 dist-test-slave-t3q3 krb5kdc[24310](info): commencing operation
krb5kdc: starting...
W20260806 16:17:42.185148 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:17:42 dist-test-slave-t3q3 krb5kdc[24310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:17:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:17:42.344008 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:43267
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:43267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:42.490957 24331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:42.491341 24331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:42.491390 24331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:42.495844 24331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:42.495929 24331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:42.495949 24331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:42.495967 24331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:42.495985 24331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:17:42.502063 24331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:43267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:43267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:42.503578 24331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:42.504946 24331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:42.515703 24338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:42.515703 24337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:42.515703 24341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:42.516222 24331 server_base.cc:1034] running on GCE node
I20260806 16:17:42.517067 24331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:42.518597 24331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:42.519804 24331 hybrid_clock.cc:648] HybridClock initialized: now 1786033062519771 us; error 45 us; skew 500 ppm
Aug 06 16:17:42 dist-test-slave-t3q3 krb5kdc[24310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033062, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:42.528455 24331 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:17:42.529865 24331 webserver.cc:460] Webserver started at http://127.11.245.190:46233/ using document root <none> and password file <none>
I20260806 16:17:42.530696 24331 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:42.530808 24331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:42.531111 24331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:42.534152 24331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "9d083979379e440daefeab8136c1cd65"
format_stamp: "Formatted at 2026-08-06 16:17:42 on dist-test-slave-t3q3"
server_key: "15f3c3664afbf59343754e3f1943821e"
server_key_iv: "ece2fe465529a233337c3e6f68f71bb7"
server_key_version: "encryptionkey@0"
I20260806 16:17:42.534849 24331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "9d083979379e440daefeab8136c1cd65"
format_stamp: "Formatted at 2026-08-06 16:17:42 on dist-test-slave-t3q3"
server_key: "15f3c3664afbf59343754e3f1943821e"
server_key_iv: "ece2fe465529a233337c3e6f68f71bb7"
server_key_version: "encryptionkey@0"
I20260806 16:17:42.539884 24331 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 16:17:42.543045 24350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:42.544713 24331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:17:42.544884 24331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "9d083979379e440daefeab8136c1cd65"
format_stamp: "Formatted at 2026-08-06 16:17:42 on dist-test-slave-t3q3"
server_key: "15f3c3664afbf59343754e3f1943821e"
server_key_iv: "ece2fe465529a233337c3e6f68f71bb7"
server_key_version: "encryptionkey@0"
I20260806 16:17:42.545035 24331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:42.563349 24331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:42.567730 24331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:42.568022 24331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:42.577557 24331 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:43267
I20260806 16:17:42.577621 24420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:43267 every 8 connection(s)
I20260806 16:17:42.578866 24331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:17:42.582752 24422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:17:42.584883 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 24331
I20260806 16:17:42.585047 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:17:42.585391 12246 external_mini_cluster.cc:1428] Setting key 3fd9e94c60d1dfb9695f64153369a834
I20260806 16:17:42.591094 24422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65: Bootstrap starting.
I20260806 16:17:42.594662 24422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65: Neither blocks nor log segments found. Creating new log.
I20260806 16:17:42.595877 24422 log.cc:824] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:17:42 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:42.598488 24422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65: No bootstrap required, opened a new log
I20260806 16:17:42.602414 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:42.587272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49206 (local address 127.11.245.190:43267)
0806 16:17:42.587837 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:42.587856 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:42.587911 (+ 55us) server_negotiation.cc:407] Connection header received
0806 16:17:42.588788 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:42.588810 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:42.589244 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:42.589600 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:42.590674 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:42.591853 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:42.593128 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:42.593640 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:42.593772 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:42.597108 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:42.597114 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:42.597129 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:42.597158 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:42.599124 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:42.599633 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:42.599639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:42.599645 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:42.599759 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:42.600116 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:42.600124 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:42.600125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:42.600607 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:42.600783 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:42.601122 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 16:17:42.601443 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":185,"threads_started":1}
I20260806 16:17:42.603382 24422 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d083979379e440daefeab8136c1cd65" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43267 } }
I20260806 16:17:42.603698 24422 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:17:42.603793 24422 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d083979379e440daefeab8136c1cd65, State: Initialized, Role: FOLLOWER
I20260806 16:17:42.604367 24422 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d083979379e440daefeab8136c1cd65" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43267 } }
I20260806 16:17:42.604554 24422 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:17:42.604642 24422 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:17:42.604773 24422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:17:42.606029 24422 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d083979379e440daefeab8136c1cd65" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43267 } }
I20260806 16:17:42.606465 24422 leader_election.cc:302] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d083979379e440daefeab8136c1cd65; no voters:
I20260806 16:17:42.606907 24422 leader_election.cc:288] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:17:42.607127 24429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:17:42.607445 24429 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 1 LEADER]: Becoming Leader. State: Replica: 9d083979379e440daefeab8136c1cd65, State: Running, Role: LEADER
I20260806 16:17:42.607797 24429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d083979379e440daefeab8136c1cd65" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43267 } }
I20260806 16:17:42.608645 24422 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:42.610491 24430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d083979379e440daefeab8136c1cd65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d083979379e440daefeab8136c1cd65" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43267 } } }
I20260806 16:17:42.610533 24431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d083979379e440daefeab8136c1cd65. Latest consensus state: current_term: 1 leader_uuid: "9d083979379e440daefeab8136c1cd65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d083979379e440daefeab8136c1cd65" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43267 } } }
I20260806 16:17:42.610685 24430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:42.610683 24431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:42.611212 24443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:42.615561 24443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:42.623611 24443 catalog_manager.cc:1347] Generated new cluster ID: 7e1e651443a748acabf3dcfbb4a47711
I20260806 16:17:42.623745 24443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:42.637395 24443 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:17:42.639623 24443 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:42.654433 24443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65: Generated new TSK 0
I20260806 16:17:42.655499 24443 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:17:42.695649 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43267
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:42.844990 24458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:42.845345 24458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:42.845395 24458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:42.849761 24458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:42.849850 24458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:42.849922 24458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:17:42.856207 24458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:42.857723 24458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:42.859040 24458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:42.872270 24466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:42.872192 24468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:42.872210 24471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:42.872758 24458 server_base.cc:1034] running on GCE node
I20260806 16:17:42.873437 24458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:42.874321 24458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:42.875524 24458 hybrid_clock.cc:648] HybridClock initialized: now 1786033062875487 us; error 52 us; skew 500 ppm
Aug 06 16:17:42 dist-test-slave-t3q3 krb5kdc[24310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033062, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:42.879194 24458 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:17:42.880702 24458 webserver.cc:460] Webserver started at http://127.11.245.129:36049/ using document root <none> and password file <none>
I20260806 16:17:42.881626 24458 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:42.881706 24458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:42.882009 24458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:42.884864 24458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "78344bbed5eb4aee80428329b92898cc"
format_stamp: "Formatted at 2026-08-06 16:17:42 on dist-test-slave-t3q3"
server_key: "0dae349db7abc52e9e663fd2f5c49ba3"
server_key_iv: "d56dbf7bb7bd5333bf20217561f17cca"
server_key_version: "encryptionkey@0"
I20260806 16:17:42.885581 24458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "78344bbed5eb4aee80428329b92898cc"
format_stamp: "Formatted at 2026-08-06 16:17:42 on dist-test-slave-t3q3"
server_key: "0dae349db7abc52e9e663fd2f5c49ba3"
server_key_iv: "d56dbf7bb7bd5333bf20217561f17cca"
server_key_version: "encryptionkey@0"
I20260806 16:17:42.890780 24458 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:17:42.894033 24479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:42.895555 24458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:42.895689 24458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "78344bbed5eb4aee80428329b92898cc"
format_stamp: "Formatted at 2026-08-06 16:17:42 on dist-test-slave-t3q3"
server_key: "0dae349db7abc52e9e663fd2f5c49ba3"
server_key_iv: "d56dbf7bb7bd5333bf20217561f17cca"
server_key_version: "encryptionkey@0"
I20260806 16:17:42.895816 24458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:42.920832 24458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:42.924988 24458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:42.925236 24458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:42.926043 24458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:42.927134 24458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:42.927213 24458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:42.927300 24458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:42.927341 24458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:42.940312 24458 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:40187
I20260806 16:17:42.940337 24654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:40187 every 8 connection(s)
I20260806 16:17:42.941540 24458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:17:42.947155 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 24458
I20260806 16:17:42.947345 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:17:42.947767 12246 external_mini_cluster.cc:1428] Setting key 27841eb79d81ef04b44c15f8dfeeb189
Aug 06 16:17:42 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033062, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:42.959340 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:42.944031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:60609 (local address 127.11.245.190:43267)
0806 16:17:42.944245 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:42.944249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:42.945267 (+ 1018us) server_negotiation.cc:407] Connection header received
0806 16:17:42.946751 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:42.946755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:42.946813 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:42.946893 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:42.948814 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:42.949594 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:42.951521 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:42.951775 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:42.951865 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:42.955301 (+ 3436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:42.955304 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:42.955307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:42.955334 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:42.956954 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:42.957541 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:42.957545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:42.957546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:42.957634 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:42.958012 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:42.958016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:42.958018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:42.958238 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:42.958349 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:42.958980 (+ 631us) server_negotiation.cc:299] Negotiation successful
0806 16:17:42.959137 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:17:42.960541 24659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:42.944541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:60609)
0806 16:17:42.945087 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:42.945136 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:42.946496 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:42.947146 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:42.947156 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:42.947780 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:42.948638 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:42.948660 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:42.949762 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:42.949767 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:42.951371 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:42.951389 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:42.951958 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:42.951965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:42.952263 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:42.953155 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:42.953176 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:42.955097 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:42.957088 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:42.957095 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:42.957115 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:42.957423 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:42.957727 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:42.957731 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:42.957733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:42.957900 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:42.958349 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:42.958358 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:42.958686 (+ 328us) client_negotiation.cc:769] Sending connection context
0806 16:17:42.958913 (+ 227us) client_negotiation.cc:240] Negotiation successful
0806 16:17:42.959329 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":136,"threads_started":1}
I20260806 16:17:42.962122 24656 heartbeater.cc:347] Connected to a master server at 127.11.245.190:43267
I20260806 16:17:42.962445 24656 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:42.963263 24656 heartbeater.cc:511] Master 127.11.245.190:43267 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
I20260806 16:17:42.965386 24373 ts_manager.cc:194] Registered new tserver with Master: 78344bbed5eb4aee80428329b92898cc (127.11.245.129:40187)
I20260806 16:17:42.967399 24373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:60609
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:17:42.985085 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43267
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:43.128485 24663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:43.128872 24663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:43.128926 24663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:43.133448 24663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:43.133541 24663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:43.133610 24663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:17:43.139760 24663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:43.141651 24663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:43.143164 24663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:43.155686 24671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:43.155675 24673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:43.155675 24670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:43.156737 24663 server_base.cc:1034] running on GCE node
I20260806 16:17:43.157272 24663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:43.158169 24663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:43.159399 24663 hybrid_clock.cc:648] HybridClock initialized: now 1786033063159371 us; error 42 us; skew 500 ppm
Aug 06 16:17:43 dist-test-slave-t3q3 krb5kdc[24310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:43.163056 24663 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:17:43.164343 24663 webserver.cc:460] Webserver started at http://127.11.245.130:41347/ using document root <none> and password file <none>
I20260806 16:17:43.165126 24663 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:43.165228 24663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:43.165524 24663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:43.168274 24663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "638da97a53e54f7087a7a237d3393f08"
format_stamp: "Formatted at 2026-08-06 16:17:43 on dist-test-slave-t3q3"
server_key: "0bbb1f8e73fc0cb6b8a2c3d61972d6c7"
server_key_iv: "cafd362961991c6f0af968408dd4dad7"
server_key_version: "encryptionkey@0"
I20260806 16:17:43.169008 24663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "638da97a53e54f7087a7a237d3393f08"
format_stamp: "Formatted at 2026-08-06 16:17:43 on dist-test-slave-t3q3"
server_key: "0bbb1f8e73fc0cb6b8a2c3d61972d6c7"
server_key_iv: "cafd362961991c6f0af968408dd4dad7"
server_key_version: "encryptionkey@0"
I20260806 16:17:43.174788 24663 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 16:17:43.178501 24686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:43.180454 24663 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:17:43.180640 24663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "638da97a53e54f7087a7a237d3393f08"
format_stamp: "Formatted at 2026-08-06 16:17:43 on dist-test-slave-t3q3"
server_key: "0bbb1f8e73fc0cb6b8a2c3d61972d6c7"
server_key_iv: "cafd362961991c6f0af968408dd4dad7"
server_key_version: "encryptionkey@0"
I20260806 16:17:43.180810 24663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:43.204854 24663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:43.209416 24663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:43.209705 24663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:43.210604 24663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:43.211802 24663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:43.211884 24663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:43.211971 24663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:43.212016 24663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:43.224756 24663 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:36357
I20260806 16:17:43.224874 24847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:36357 every 8 connection(s)
I20260806 16:17:43.226164 24663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:17:43.236197 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 24663
I20260806 16:17:43.236372 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:17:43.236810 12246 external_mini_cluster.cc:1428] Setting key 219135a459d6269c9288e9fc3358fced
Aug 06 16:17:43 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:43.243374 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:43.228862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:53161 (local address 127.11.245.190:43267)
0806 16:17:43.229059 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:43.229063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:43.230175 (+ 1112us) server_negotiation.cc:407] Connection header received
0806 16:17:43.231806 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:43.231811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:43.231889 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:43.232007 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:43.233604 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:43.234201 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:43.235493 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:43.235738 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:43.235856 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:43.239209 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:43.239213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:43.239215 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:43.239243 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:43.240968 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:43.241545 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:43.241549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:43.241551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:43.241645 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:43.242038 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:43.242043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:43.242045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:43.242264 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:43.242382 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:43.243014 (+ 632us) server_negotiation.cc:299] Negotiation successful
0806 16:17:43.243174 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 16:17:43.244601 24854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:43.229375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:53161)
0806 16:17:43.229963 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:43.230030 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:43.231533 (+ 1503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:43.232279 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:43.232288 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:43.232777 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:43.233428 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:43.233444 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:43.234350 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:43.234360 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:43.235347 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:43.235358 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:43.235936 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:43.235941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:43.236200 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:43.237080 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:43.237099 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:43.239011 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:43.241101 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:43.241108 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:43.241130 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:43.241427 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:43.241746 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:43.241750 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:43.241752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:43.241924 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:43.242396 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:43.242406 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:43.242732 (+ 326us) client_negotiation.cc:769] Sending connection context
0806 16:17:43.242968 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 16:17:43.243400 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":140,"threads_started":1}
I20260806 16:17:43.246176 24849 heartbeater.cc:347] Connected to a master server at 127.11.245.190:43267
I20260806 16:17:43.246497 24849 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:43.247176 24849 heartbeater.cc:511] Master 127.11.245.190:43267 requested a full tablet report, sending...
I20260806 16:17:43.248637 24373 ts_manager.cc:194] Registered new tserver with Master: 638da97a53e54f7087a7a237d3393f08 (127.11.245.130:36357)
I20260806 16:17:43.249380 24373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:53161
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:17:43.276985 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43267
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:43.430752 24858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:43.431170 24858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:43.431259 24858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:43.435775 24858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:43.435906 24858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:43.436014 24858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:17:43.442034 24858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:43.443614 24858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:43.445053 24858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:43.457798 24865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:43.457881 24867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:43.457876 24858 server_base.cc:1034] running on GCE node
W20260806 16:17:43.458148 24870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:43.458700 24858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:43.459786 24858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:43.461063 24858 hybrid_clock.cc:648] HybridClock initialized: now 1786033063461036 us; error 56 us; skew 500 ppm
Aug 06 16:17:43 dist-test-slave-t3q3 krb5kdc[24310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:43.465997 24858 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:17:43.467384 24858 webserver.cc:460] Webserver started at http://127.11.245.131:44379/ using document root <none> and password file <none>
I20260806 16:17:43.468333 24858 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:43.468446 24858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:43.468942 24858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:17:43.472007 24858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "5b64c995215949d68eaa04e3b39b6abc"
format_stamp: "Formatted at 2026-08-06 16:17:43 on dist-test-slave-t3q3"
server_key: "b1d789311cd2457d400827ee3c43a03b"
server_key_iv: "185d312ca36796dd4ed6bc257fc05046"
server_key_version: "encryptionkey@0"
I20260806 16:17:43.472903 24858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "5b64c995215949d68eaa04e3b39b6abc"
format_stamp: "Formatted at 2026-08-06 16:17:43 on dist-test-slave-t3q3"
server_key: "b1d789311cd2457d400827ee3c43a03b"
server_key_iv: "185d312ca36796dd4ed6bc257fc05046"
server_key_version: "encryptionkey@0"
I20260806 16:17:43.478374 24858 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:17:43.481740 24880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:43.483491 24858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 16:17:43.483678 24858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "5b64c995215949d68eaa04e3b39b6abc"
format_stamp: "Formatted at 2026-08-06 16:17:43 on dist-test-slave-t3q3"
server_key: "b1d789311cd2457d400827ee3c43a03b"
server_key_iv: "185d312ca36796dd4ed6bc257fc05046"
server_key_version: "encryptionkey@0"
I20260806 16:17:43.483847 24858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:43.499022 24858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:43.503176 24858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:43.503496 24858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:43.504357 24858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:43.505498 24858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:43.505589 24858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:43.505674 24858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:43.505729 24858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:43.518451 24858 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:35495
I20260806 16:17:43.518471 25045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:35495 every 8 connection(s)
I20260806 16:17:43.519758 24858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:17:43.529603 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 24858
I20260806 16:17:43.529821 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:17:43.530218 12246 external_mini_cluster.cc:1428] Setting key 9bfda31b36f86f576a220dc416698a11
Aug 06 16:17:43 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:17:43.536991 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:43.522386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:40707 (local address 127.11.245.190:43267)
0806 16:17:43.522576 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:43.522580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:43.524002 (+ 1422us) server_negotiation.cc:407] Connection header received
0806 16:17:43.525135 (+ 1133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:43.525139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:43.525208 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:43.525320 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:43.526900 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:43.527633 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:43.529062 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:43.529433 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:43.529616 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:43.532918 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:43.532921 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:43.532923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:43.532972 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:43.534371 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:43.534969 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:43.534973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:43.534974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:43.535066 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:43.535672 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:43.535676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:43.535677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:43.535907 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:43.536002 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:43.536701 (+ 699us) server_negotiation.cc:299] Negotiation successful
0806 16:17:43.536835 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 16:17:43.538128 25053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:43.522798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:40707)
0806 16:17:43.523762 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:43.523813 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:43.524865 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:43.525551 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:43.525559 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:43.526010 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:43.526687 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:43.526704 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:43.527804 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:43.527807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:43.528922 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:43.528933 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:43.529674 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:43.529677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:43.529962 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:43.530735 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:43.530757 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:43.532695 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:17:43.534491 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:43.534497 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:43.534515 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:43.534805 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:43.535318 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:17:43.535321 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:17:43.535323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:17:43.535529 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:17:43.536019 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:17:43.536025 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:17:43.536410 (+ 385us) client_negotiation.cc:769] Sending connection context
0806 16:17:43.536628 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 16:17:43.536987 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":152,"threads_started":1}
I20260806 16:17:43.539592 25046 heartbeater.cc:347] Connected to a master server at 127.11.245.190:43267
I20260806 16:17:43.539907 25046 heartbeater.cc:464] Registering TS with master...
I20260806 16:17:43.540762 25046 heartbeater.cc:511] Master 127.11.245.190:43267 requested a full tablet report, sending...
I20260806 16:17:43.542163 24373 ts_manager.cc:194] Registered new tserver with Master: 5b64c995215949d68eaa04e3b39b6abc (127.11.245.131:35495)
I20260806 16:17:43.542860 24373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:40707
I20260806 16:17:43.545181 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.129.keytab.
Entry for principal oryx/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.129.keytab.
WARNING: no policy specified for oryx/127.11.245.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.245.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.130.keytab.
Entry for principal oryx/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.130.keytab.
WARNING: no policy specified for oryx/127.11.245.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.245.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.131.keytab.
Entry for principal oryx/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.131.keytab.
I20260806 16:17:43.653788 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 24458
I20260806 16:17:43.681154 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 24663
I20260806 16:17:43.711490 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 24858
I20260806 16:17:43.734309 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 24331
I20260806 16:17:43.754320 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:43267
--webserver_interface=127.11.245.190
--webserver_port=46233
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:43267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:17:43.901684 25061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:43.902067 25061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:43.902115 25061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:43.906519 25061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:17:43.906612 25061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:43.906634 25061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:43.906653 25061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:17:43.906669 25061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:17:43.912834 25061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:43267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:43267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=46233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:43.914405 25061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:43.915787 25061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:43.927603 25067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:43.927568 25069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:43.927701 25071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:43.928028 25061 server_base.cc:1034] running on GCE node
I20260806 16:17:43.928810 25061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:43.930413 25061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:43.931666 25061 hybrid_clock.cc:648] HybridClock initialized: now 1786033063931631 us; error 55 us; skew 500 ppm
Aug 06 16:17:43 dist-test-slave-t3q3 krb5kdc[24310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:43.935405 25061 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:17:43.936818 25061 webserver.cc:460] Webserver started at http://127.11.245.190:46233/ using document root <none> and password file <none>
I20260806 16:17:43.937636 25061 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:43.937772 25061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:43.943012 25061 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 16:17:43.945945 25080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:43.947728 25061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 16:17:43.947978 25061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "9d083979379e440daefeab8136c1cd65"
format_stamp: "Formatted at 2026-08-06 16:17:42 on dist-test-slave-t3q3"
server_key: "15f3c3664afbf59343754e3f1943821e"
server_key_iv: "ece2fe465529a233337c3e6f68f71bb7"
server_key_version: "encryptionkey@0"
I20260806 16:17:43.948843 25061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:43.970778 25061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:43.975572 25061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:43.975867 25061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:43.985172 25061 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:43267
I20260806 16:17:43.985203 25147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:43267 every 8 connection(s)
I20260806 16:17:43.986649 25061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:17:43.993148 25148 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:17:43.995134 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 25061
I20260806 16:17:43.995787 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:40187
--local_ip_for_outbound_sockets=127.11.245.129
--tserver_master_addrs=127.11.245.190:43267
--webserver_port=36049
--webserver_interface=127.11.245.129
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.245.129
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:17:43.995805 25148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65: Bootstrap starting.
I20260806 16:17:44.019305 25148 log.cc:824] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65: Log is configured to *not* fsync() on all Append() calls
I20260806 16:17:44.028669 25148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:17:44.029202 25148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65: Bootstrap complete.
I20260806 16:17:44.035192 25148 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d083979379e440daefeab8136c1cd65" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43267 } }
I20260806 16:17:44.035773 25148 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d083979379e440daefeab8136c1cd65, State: Initialized, Role: FOLLOWER
I20260806 16:17:44.036553 25148 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d083979379e440daefeab8136c1cd65" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43267 } }
I20260806 16:17:44.036761 25148 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:17:44.036862 25148 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:17:44.037002 25148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 1 FOLLOWER]: Advancing to term 2
I20260806 16:17:44.039968 25148 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d083979379e440daefeab8136c1cd65" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43267 } }
I20260806 16:17:44.040508 25148 leader_election.cc:302] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d083979379e440daefeab8136c1cd65; no voters:
I20260806 16:17:44.040961 25148 leader_election.cc:288] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 16:17:44.041131 25156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 16:17:44.041479 25156 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [term 2 LEADER]: Becoming Leader. State: Replica: 9d083979379e440daefeab8136c1cd65, State: Running, Role: LEADER
I20260806 16:17:44.041889 25156 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d083979379e440daefeab8136c1cd65" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43267 } }
I20260806 16:17:44.042665 25148 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:17:44.043695 25157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9d083979379e440daefeab8136c1cd65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d083979379e440daefeab8136c1cd65" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43267 } } }
I20260806 16:17:44.043848 25157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:44.044243 25158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d083979379e440daefeab8136c1cd65. Latest consensus state: current_term: 2 leader_uuid: "9d083979379e440daefeab8136c1cd65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d083979379e440daefeab8136c1cd65" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43267 } } }
I20260806 16:17:44.044349 25158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65 [sys.catalog]: This master's current role is: LEADER
I20260806 16:17:44.045343 25165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:17:44.049856 25165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:17:44.050673 25165 catalog_manager.cc:1259] Loaded cluster ID: 7e1e651443a748acabf3dcfbb4a47711
I20260806 16:17:44.050747 25165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:17:44.053814 25165 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:17:44.054699 25165 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9d083979379e440daefeab8136c1cd65: Loaded TSK: 0
I20260806 16:17:44.056339 25165 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 16:17:44.151944 25153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:44.152453 25153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:44.152570 25153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:44.157262 25153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:44.157371 25153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:44.157500 25153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:17:44.163564 25153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.129.keytab
--principal=oryx/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:40187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=36049
--tserver_master_addrs=127.11.245.190:43267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:44.165146 25153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:44.166517 25153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:44.180465 25189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:44.180505 25190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:44.180497 25194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:44.181067 25153 server_base.cc:1034] running on GCE node
I20260806 16:17:44.182053 25153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:44.182880 25153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:44.184135 25153 hybrid_clock.cc:648] HybridClock initialized: now 1786033064184109 us; error 41 us; skew 500 ppm
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033064, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:44.187786 25153 init.cc:377] Logged in from keytab as oryx/127.11.245.129@KRBTEST.COM (short username oryx)
I20260806 16:17:44.189029 25153 webserver.cc:460] Webserver started at http://127.11.245.129:36049/ using document root <none> and password file <none>
I20260806 16:17:44.189836 25153 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:44.189980 25153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:44.196153 25153 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.007s
I20260806 16:17:44.199494 25202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:44.201535 25153 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 16:17:44.201824 25153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "78344bbed5eb4aee80428329b92898cc"
format_stamp: "Formatted at 2026-08-06 16:17:42 on dist-test-slave-t3q3"
server_key: "0dae349db7abc52e9e663fd2f5c49ba3"
server_key_iv: "d56dbf7bb7bd5333bf20217561f17cca"
server_key_version: "encryptionkey@0"
I20260806 16:17:44.202625 25153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:44.233490 25153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:44.237586 25153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:44.237843 25153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:44.238636 25153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:44.239732 25153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:44.239810 25153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:44.239895 25153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:44.239953 25153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:44.252233 25153 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:40187
I20260806 16:17:44.252285 25363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:40187 every 8 connection(s)
I20260806 16:17:44.253811 25153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:17:44.257894 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 25153
I20260806 16:17:44.258558 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:36357
--local_ip_for_outbound_sockets=127.11.245.130
--tserver_master_addrs=127.11.245.190:43267
--webserver_port=41347
--webserver_interface=127.11.245.130
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.245.130
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 16:17:44.273308 25367 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:44.257030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:36403 (local address 127.11.245.190:43267)
0806 16:17:44.257671 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:44.257693 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:44.258336 (+ 643us) server_negotiation.cc:407] Connection header received
0806 16:17:44.260591 (+ 2255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:44.260603 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:44.261117 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:44.261593 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:44.263047 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.264286 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.266342 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.266881 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.267032 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.272337 (+ 5305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:36403: BlockingRecv error: recv got EOF from 127.11.245.129:36403 (error 108)
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":172,"threads_started":1}
I20260806 16:17:44.273353 25369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:44.257116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:36403)
0806 16:17:44.258074 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:44.258159 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:44.259924 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:44.261716 (+ 1792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:44.261725 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:44.262217 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:44.262814 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.262831 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.264453 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.264459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.266136 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.266153 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.267102 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.267109 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.267353 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.268216 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:44.268249 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:44.272005 (+ 3756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":226,"threads_started":1}
W20260806 16:17:44.274168 25364 heartbeater.cc:644] Failed to heartbeat to 127.11.245.190:43267 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.245.190:43267: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:44.281121 25369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:44.274836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:42999)
0806 16:17:44.274996 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:44.275014 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:44.275119 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:44.275425 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:44.275428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:44.275670 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:44.275818 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.275825 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.276700 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.276704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.277306 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.277315 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.277805 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.277809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.277906 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.278488 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:44.278514 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:44.280952 (+ 2438us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 16:17:44.281316 25367 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:44.274941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:42999 (local address 127.11.245.190:43267)
0806 16:17:44.275082 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:44.275086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:44.275101 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:17:44.275249 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:44.275253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:44.275311 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:44.275404 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:44.275943 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.276579 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.277439 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.277680 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.277792 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.281128 (+ 3336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:42999: BlockingRecv error: recv got EOF from 127.11.245.129:42999 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:44.287693 25369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:44.281797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:58759)
0806 16:17:44.282086 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:44.282100 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:44.282186 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:44.282479 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:44.282482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:44.282643 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:44.282731 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.282736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.283803 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.283807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.284438 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.284449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.285047 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.285049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.285115 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.285679 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:44.285691 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:44.287571 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215}
W20260806 16:17:44.287876 25367 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:44.281946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:58759 (local address 127.11.245.190:43267)
0806 16:17:44.282104 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:44.282109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:44.282145 (+ 36us) server_negotiation.cc:407] Connection header received
0806 16:17:44.282275 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:44.282278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:44.282331 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:44.282419 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:44.282853 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.283671 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.284555 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.284922 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.285029 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.287742 (+ 2713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:58759: BlockingRecv error: recv got EOF from 127.11.245.129:58759 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 16:17:44.288056 25364 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:17:44.410310 25370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:44.410686 25370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:44.410738 25370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:44.415405 25370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:44.415511 25370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:44.415578 25370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:17:44.421869 25370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.130.keytab
--principal=oryx/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:36357
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=41347
--tserver_master_addrs=127.11.245.190:43267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:44.423297 25370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:44.424609 25370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:44.436619 25380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:44.436619 25379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:44.436622 25384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:44.437043 25370 server_base.cc:1034] running on GCE node
I20260806 16:17:44.437506 25370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:44.438292 25370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:44.439502 25370 hybrid_clock.cc:648] HybridClock initialized: now 1786033064439462 us; error 49 us; skew 500 ppm
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033064, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:44.443166 25370 init.cc:377] Logged in from keytab as oryx/127.11.245.130@KRBTEST.COM (short username oryx)
I20260806 16:17:44.444638 25370 webserver.cc:460] Webserver started at http://127.11.245.130:41347/ using document root <none> and password file <none>
I20260806 16:17:44.445431 25370 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:44.445571 25370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:44.450807 25370 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 16:17:44.453660 25393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:44.455250 25370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 16:17:44.455405 25370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "638da97a53e54f7087a7a237d3393f08"
format_stamp: "Formatted at 2026-08-06 16:17:43 on dist-test-slave-t3q3"
server_key: "0bbb1f8e73fc0cb6b8a2c3d61972d6c7"
server_key_iv: "cafd362961991c6f0af968408dd4dad7"
server_key_version: "encryptionkey@0"
I20260806 16:17:44.456141 25370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:44.472221 25370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:44.475811 25370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:44.476145 25370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:44.476928 25370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:44.477984 25370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:44.478068 25370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:44.478166 25370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:44.478211 25370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:44.489869 25370 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:36357
I20260806 16:17:44.489907 25554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:36357 every 8 connection(s)
I20260806 16:17:44.491364 25370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:17:44.500690 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 25370
I20260806 16:17:44.501365 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:35495
--local_ip_for_outbound_sockets=127.11.245.131
--tserver_master_addrs=127.11.245.190:43267
--webserver_port=44379
--webserver_interface=127.11.245.131
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.245.131
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 16:17:44.505667 25367 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:44.494089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:47543 (local address 127.11.245.190:43267)
0806 16:17:44.494278 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:44.494284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:44.495413 (+ 1129us) server_negotiation.cc:407] Connection header received
0806 16:17:44.496963 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:44.496966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:44.497032 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:44.497112 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:44.498508 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.499094 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.500454 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.500815 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.500958 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.505528 (+ 4570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:47543: BlockingRecv error: recv got EOF from 127.11.245.130:47543 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 16:17:44.506760 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:44.494637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:47543)
0806 16:17:44.495232 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:44.495280 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:44.496733 (+ 1453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:44.497309 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:44.497316 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:44.497751 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:44.498358 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.498374 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.499214 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.499217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.500300 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.500311 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.501030 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.501034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.501194 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.501977 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:44.502009 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:44.505377 (+ 3368us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":166,"threads_started":1}
W20260806 16:17:44.507417 25555 heartbeater.cc:644] Failed to heartbeat to 127.11.245.190:43267 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.245.190:43267: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:44.515574 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:44.507972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:39765)
0806 16:17:44.508127 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:44.508142 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:44.508262 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:44.508598 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:44.508603 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:44.508817 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:44.509004 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.509014 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.509810 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.509816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.510846 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.510862 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.511311 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.511317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.511451 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.512044 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:44.512086 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:44.515364 (+ 3278us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 16:17:44.515689 25367 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:44.508119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:39765 (local address 127.11.245.190:43267)
0806 16:17:44.508270 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:44.508275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:44.508291 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:17:44.508354 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:44.508356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:44.508431 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:44.508526 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:44.509136 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.509690 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.510977 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.511190 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.511266 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.515539 (+ 4273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:39765: BlockingRecv error: recv got EOF from 127.11.245.130:39765 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:44.522316 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:44.516520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:52569)
0806 16:17:44.516638 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:44.516655 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:44.516753 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:44.517039 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:44.517043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:44.517225 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:44.517312 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.517326 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.518162 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.518167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.518791 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.518799 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.519305 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.519310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.519416 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.520012 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:44.520026 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:44.522148 (+ 2122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 16:17:44.522392 25367 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:44.516621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:52569 (local address 127.11.245.190:43267)
0806 16:17:44.516747 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:44.516752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:44.516768 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:17:44.516855 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:44.516858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:44.516916 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:44.517014 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:44.517452 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.518039 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.518930 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.519181 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.519300 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.522266 (+ 2966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:52569: BlockingRecv error: recv got EOF from 127.11.245.130:52569 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 16:17:44.522755 25555 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:17:44.650949 25558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:17:44.651294 25558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:17:44.651343 25558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:17:44.655874 25558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:17:44.655963 25558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:17:44.656029 25558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:17:44.662747 25558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/krb5kdc/oryx_127.11.245.131.keytab
--principal=oryx/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:35495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=44379
--tserver_master_addrs=127.11.245.190:43267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:17:44.664316 25558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:17:44.665647 25558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:17:44.678133 25564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:44.678122 25565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:17:44.678120 25570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:17:44.678437 25558 server_base.cc:1034] running on GCE node
I20260806 16:17:44.679080 25558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:17:44.679908 25558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:17:44.681093 25558 hybrid_clock.cc:648] HybridClock initialized: now 1786033064681039 us; error 72 us; skew 500 ppm
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033064, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:17:44.684756 25558 init.cc:377] Logged in from keytab as oryx/127.11.245.131@KRBTEST.COM (short username oryx)
I20260806 16:17:44.685985 25558 webserver.cc:460] Webserver started at http://127.11.245.131:44379/ using document root <none> and password file <none>
I20260806 16:17:44.686759 25558 fs_manager.cc:359] Metadata directory not provided
I20260806 16:17:44.686877 25558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:17:44.691828 25558 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:17:44.694656 25583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:44.696409 25558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:17:44.696581 25558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "5b64c995215949d68eaa04e3b39b6abc"
format_stamp: "Formatted at 2026-08-06 16:17:43 on dist-test-slave-t3q3"
server_key: "b1d789311cd2457d400827ee3c43a03b"
server_key_iv: "185d312ca36796dd4ed6bc257fc05046"
server_key_version: "encryptionkey@0"
I20260806 16:17:44.697289 25558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:17:44.723284 25558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:17:44.727275 25558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:17:44.727555 25558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:17:44.728470 25558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:17:44.729982 25558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:17:44.730062 25558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:44.730141 25558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:17:44.730175 25558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:17:44.743336 25558 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:35495
I20260806 16:17:44.743397 25735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:35495 every 8 connection(s)
I20260806 16:17:44.744813 25558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:17:44.753870 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 25558
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 16:17:44.761106 25367 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:44.747251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:50633 (local address 127.11.245.190:43267)
0806 16:17:44.747435 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:44.747440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:44.748858 (+ 1418us) server_negotiation.cc:407] Connection header received
0806 16:17:44.751162 (+ 2304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:44.751166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:44.751239 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:44.751325 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:44.753313 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.754214 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.756255 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.756484 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.756575 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.760927 (+ 4352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:50633: BlockingRecv error: recv got EOF from 127.11.245.131:50633 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 16:17:44.762046 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:44.747849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:50633)
0806 16:17:44.748638 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:44.748706 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:44.750740 (+ 2034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:44.751565 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:44.751574 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:44.752139 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:44.752944 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.752964 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.754468 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.754471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.756121 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.756138 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.756679 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.756684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.756939 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.757752 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:44.757771 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:44.760756 (+ 2985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":206,"threads_started":1}
W20260806 16:17:44.762653 25736 heartbeater.cc:644] Failed to heartbeat to 127.11.245.190:43267 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.245.190:43267: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
I20260806 16:17:44.765589 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:44.755414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49210 (local address 127.11.245.190:43267)
0806 16:17:44.755723 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:44.755727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:44.755739 (+ 12us) server_negotiation.cc:407] Connection header received
0806 16:17:44.755778 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:44.755780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:44.755826 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:44.755911 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:44.756945 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.757510 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.758452 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.758704 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.758788 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.760510 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:17:44.760518 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:17:44.760539 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:17:44.760569 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:17:44.762927 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:44.763566 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:44.763573 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:44.763578 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:44.763776 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:17:44.764105 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:17:44.764111 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:17:44.764115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:17:44.764562 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:17:44.764736 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 16:17:44.765116 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 16:17:44.765399 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":145,"threads_started":1}
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:44.768918 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:44.763214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:60399)
0806 16:17:44.763324 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:44.763341 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:44.763418 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:44.763686 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:44.763689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:44.763875 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:44.763994 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.764000 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.764845 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.764848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.765674 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.765687 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.766144 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.766147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.766249 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.766710 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:44.766721 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:44.768757 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 16:17:44.768993 25367 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:44.763305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:60399 (local address 127.11.245.190:43267)
0806 16:17:44.763461 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:44.763468 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:44.763482 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:17:44.763520 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:44.763522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:44.763571 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:44.763634 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:44.764155 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.764724 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.765782 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.766023 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.766137 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.768876 (+ 2739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:60399: BlockingRecv error: recv got EOF from 127.11.245.131:60399 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:44 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:44.774863 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:44.769664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:36775)
0806 16:17:44.769785 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:44.769800 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:44.769901 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:44.770132 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:44.770134 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:44.770271 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:44.770358 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.770368 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.771108 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.771110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.771668 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:44.771677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.772178 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.772181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:44.772243 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.772702 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:44.772714 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:44.774732 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 16:17:44.775158 25367 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:44.769719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:36775 (local address 127.11.245.190:43267)
0806 16:17:44.769872 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:44.769876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:44.769888 (+ 12us) server_negotiation.cc:407] Connection header received
0806 16:17:44.769987 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:44.769988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:44.770028 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:44.770095 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:44.770473 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.770981 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.771789 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:44.771999 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:44.772164 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:44.775028 (+ 2864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:36775: BlockingRecv error: recv got EOF from 127.11.245.131:36775 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260806 16:17:44.775158 25736 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 16:17:45 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:45 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:45.296855 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:45.289169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:38677)
0806 16:17:45.289677 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:45.289697 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:45.289874 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:45.290296 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:45.290299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:45.290639 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:45.290868 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:45.290879 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:45.291984 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:45.291988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:45.292766 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:45.292778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:45.293317 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:45.293320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:45.293412 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:45.294145 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:45.294158 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:45.296682 (+ 2524us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":125,"threads_started":1}
W20260806 16:17:45.297115 25745 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:45.289287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:38677 (local address 127.11.245.190:43267)
0806 16:17:45.289638 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:45.289643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:45.289830 (+ 187us) server_negotiation.cc:407] Connection header received
0806 16:17:45.290035 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:45.290039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:45.290120 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:45.290232 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:45.291043 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:45.291844 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:45.292918 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:45.293177 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:45.293295 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:45.296919 (+ 3624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:38677: BlockingRecv error: recv got EOF from 127.11.245.129:38677 (error 108)
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":118,"threads_started":1}
Aug 06 16:17:45 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:45 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:45.530663 25747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:45.523828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:56745)
0806 16:17:45.524260 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:45.524277 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:45.524410 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:45.524771 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:45.524777 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:45.525043 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:45.525177 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:45.525185 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:45.526075 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:45.526081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:45.526792 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:45.526804 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:45.527391 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:45.527396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:45.527522 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:45.528240 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:45.528253 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:45.530501 (+ 2248us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":144,"threads_started":1}
W20260806 16:17:45.530990 25745 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:45.524049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:56745 (local address 127.11.245.190:43267)
0806 16:17:45.524356 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:45.524361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:45.524382 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:17:45.524542 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:45.524545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:45.524605 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:45.524699 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:45.525316 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:45.525912 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:45.526927 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:45.527251 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:45.527369 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:45.530668 (+ 3299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:56745: BlockingRecv error: recv got EOF from 127.11.245.130:56745 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 06 16:17:45 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:45 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:45.783072 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:45.776037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:56515)
0806 16:17:45.776494 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:45.776509 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:45.776642 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:45.777056 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:45.777059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:45.777297 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:45.777466 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:45.777473 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:45.778387 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:45.778390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:45.779028 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:45.779040 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:45.779612 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:45.779616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:45.779694 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:45.780341 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:45.780361 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:45.782867 (+ 2506us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":182,"threads_started":1}
W20260806 16:17:45.783252 25745 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:45.776130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:56515 (local address 127.11.245.190:43267)
0806 16:17:45.776382 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:45.776390 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:45.776621 (+ 231us) server_negotiation.cc:407] Connection header received
0806 16:17:45.776796 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:45.776799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:45.776860 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:45.776952 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:45.777622 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:45.778225 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:45.779194 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:45.779473 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:45.779568 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:45.783095 (+ 3527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:56515: BlockingRecv error: recv got EOF from 127.11.245.131:56515 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 16:17:46 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:46 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:46.304816 25752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:46.298170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:38759)
0806 16:17:46.298527 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:46.298544 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:46.298680 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:46.298977 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:46.298979 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:46.299217 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:46.299347 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:46.299353 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:46.300264 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:46.300267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:46.300932 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:46.300942 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:46.301431 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:46.301434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:46.301505 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:46.302279 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:46.302296 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:46.304644 (+ 2348us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":125,"threads_started":1}
W20260806 16:17:46.305174 25753 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:46.298230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:38759 (local address 127.11.245.190:43267)
0806 16:17:46.298571 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:46.298576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:46.298629 (+ 53us) server_negotiation.cc:407] Connection header received
0806 16:17:46.298776 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:46.298778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:46.298834 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:46.298936 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:46.299492 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:46.300148 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:46.301059 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:46.301298 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:46.301383 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:46.305001 (+ 3618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:38759: BlockingRecv error: recv got EOF from 127.11.245.129:38759 (error 108)
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":153,"threads_started":1}
Aug 06 16:17:46 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:46 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:46.538640 25754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:46.531801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:51939)
0806 16:17:46.532224 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:46.532240 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:46.532381 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:46.532693 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:46.532695 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:46.532933 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:46.533084 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:46.533092 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:46.533929 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:46.533932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:46.534545 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:46.534555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:46.534997 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:46.535000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:46.535080 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:46.535733 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:46.535748 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:46.538433 (+ 2685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":142,"threads_started":1}
W20260806 16:17:46.538702 25753 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:46.531927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:51939 (local address 127.11.245.190:43267)
0806 16:17:46.532155 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:46.532159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:46.532379 (+ 220us) server_negotiation.cc:407] Connection header received
0806 16:17:46.532482 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:46.532498 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:46.532567 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:46.532648 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:46.533214 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:46.533818 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:46.534662 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:46.534888 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:46.534973 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:46.538590 (+ 3617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:51939: BlockingRecv error: recv got EOF from 127.11.245.130:51939 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 16:17:46 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:46 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:46.791846 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:46.784434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:36399)
0806 16:17:46.784828 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:46.784845 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:46.785022 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:46.785580 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:46.785583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:46.785846 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:46.786076 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:46.786087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:46.786976 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:46.786980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:46.787618 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:46.787630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:46.788226 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:46.788229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:46.788313 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:46.788969 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:46.788984 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:46.791534 (+ 2550us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":159,"threads_started":1}
W20260806 16:17:46.791970 25753 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:46.784631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:36399 (local address 127.11.245.190:43267)
0806 16:17:46.784813 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:46.784819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:46.784942 (+ 123us) server_negotiation.cc:407] Connection header received
0806 16:17:46.785200 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:46.785203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:46.785378 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:46.785472 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:46.786224 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:46.786845 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:46.787773 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:46.788025 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:46.788284 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:46.791793 (+ 3509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:36399: BlockingRecv error: recv got EOF from 127.11.245.131:36399 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 16:17:47 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:47 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:47.313544 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:47.305885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:42731)
0806 16:17:47.306288 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:47.306305 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:47.306454 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:47.306887 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:47.306890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:47.307139 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:47.307274 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:47.307281 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:47.308375 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:47.308380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:47.309132 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:47.309148 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:47.309759 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:47.309763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:47.309854 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:47.310630 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:47.310644 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:47.313362 (+ 2718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":108,"threads_started":1}
W20260806 16:17:47.313884 25759 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:47.305932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:42731 (local address 127.11.245.190:43267)
0806 16:17:47.306300 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:47.306305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:47.306390 (+ 85us) server_negotiation.cc:407] Connection header received
0806 16:17:47.306601 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:47.306603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:47.306668 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:47.306840 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:47.307438 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:47.308186 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:47.309289 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:47.309593 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:47.309695 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:47.313698 (+ 4003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:42731: BlockingRecv error: recv got EOF from 127.11.245.129:42731 (error 108)
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":144,"threads_started":1}
Aug 06 16:17:47 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:47 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:47.549557 25760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:47.539659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:54897)
0806 16:17:47.539992 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:47.540014 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:47.540222 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:47.540496 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:47.540498 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:47.540741 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:47.540871 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:47.540877 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:47.541773 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:47.541775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:47.542406 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:47.542415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:47.542922 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:47.542925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:47.543020 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:47.546706 (+ 3686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:47.546729 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:47.549383 (+ 2654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":115,"threads_started":1}
W20260806 16:17:47.549741 25759 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:47.539685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:54897 (local address 127.11.245.190:43267)
0806 16:17:47.539894 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:47.539899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:47.540163 (+ 264us) server_negotiation.cc:407] Connection header received
0806 16:17:47.540308 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:47.540311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:47.540373 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:47.540449 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:47.541006 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:47.541616 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:47.542513 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:47.542749 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:47.542883 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:47.549561 (+ 6678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:54897: BlockingRecv error: recv got EOF from 127.11.245.130:54897 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 06 16:17:47 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:47 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:47.801231 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:47.793248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:45873)
0806 16:17:47.793659 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:47.793675 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:47.793818 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:47.794189 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:47.794192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:47.794451 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:47.794588 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:47.794594 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:47.795669 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:47.795673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:47.796554 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:47.796567 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:47.797180 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:47.797183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:47.797262 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:47.798000 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:47.798030 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:47.801002 (+ 2972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":140,"threads_started":1}
W20260806 16:17:47.801337 25759 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:47.793319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:45873 (local address 127.11.245.190:43267)
0806 16:17:47.793480 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:47.793486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:47.793754 (+ 268us) server_negotiation.cc:407] Connection header received
0806 16:17:47.793916 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:47.793919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:47.793990 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:47.794123 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:47.794756 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:47.795376 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:47.796719 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:47.797028 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:47.797137 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:47.801205 (+ 4068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:45873: BlockingRecv error: recv got EOF from 127.11.245.131:45873 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 16:17:48 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:48 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:48.322570 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:48.314763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:52449)
0806 16:17:48.315324 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:48.315345 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:48.315533 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:48.315923 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:48.315927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:48.316291 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:48.316469 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:48.316479 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:48.317495 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:48.317499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:48.318101 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:48.318110 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:48.318696 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:48.318700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:48.318784 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:48.319800 (+ 1016us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:48.319822 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:48.322395 (+ 2573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":121,"threads_started":1}
W20260806 16:17:48.322963 25763 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:48.314807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:52449 (local address 127.11.245.190:43267)
0806 16:17:48.315273 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:48.315279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:48.315483 (+ 204us) server_negotiation.cc:407] Connection header received
0806 16:17:48.315669 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:48.315674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:48.315755 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:48.315846 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:48.316637 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:48.317322 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:48.318257 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:48.318545 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:48.318646 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:48.322606 (+ 3960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:52449: BlockingRecv error: recv got EOF from 127.11.245.129:52449 (error 108)
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":86,"threads_started":1}
Aug 06 16:17:48 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:48 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:48.557739 25764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:48.550574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:51941)
0806 16:17:48.551040 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:48.551056 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:48.551243 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:48.551622 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:48.551625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:48.551913 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:48.552044 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:48.552052 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:48.553107 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:48.553113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:48.553784 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:48.553794 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:48.554359 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:48.554363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:48.554473 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:48.555237 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:48.555252 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:48.557512 (+ 2260us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":170,"threads_started":1}
W20260806 16:17:48.557783 25763 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:48.550674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:51941 (local address 127.11.245.190:43267)
0806 16:17:48.550912 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:48.550918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:48.551154 (+ 236us) server_negotiation.cc:407] Connection header received
0806 16:17:48.551403 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:48.551407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:48.551477 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:48.551584 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:48.552268 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:48.552975 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:48.553925 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:48.554216 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:48.554348 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:48.557664 (+ 3316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:51941: BlockingRecv error: recv got EOF from 127.11.245.130:51941 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 16:17:48 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:48 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:48.808797 25765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:48.802275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:44747)
0806 16:17:48.802626 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:48.802646 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:48.802791 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:48.803067 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:48.803070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:48.803297 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:48.803421 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:48.803426 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:48.804350 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:48.804353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:48.805203 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:48.805217 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:48.805731 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:48.805734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:48.805802 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:48.806426 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:48.806442 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:48.808650 (+ 2208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":151,"threads_started":1}
W20260806 16:17:48.808976 25763 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:48.802371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:44747 (local address 127.11.245.190:43267)
0806 16:17:48.802545 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:48.802549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:48.802734 (+ 185us) server_negotiation.cc:407] Connection header received
0806 16:17:48.802873 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:48.802875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:48.802933 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:48.803021 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:48.803540 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:48.804231 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:48.805334 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:48.805622 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:48.805709 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:48.808802 (+ 3093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:44747: BlockingRecv error: recv got EOF from 127.11.245.131:44747 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 16:17:49 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:49 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:49.330683 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:49.323846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:52381)
0806 16:17:49.324239 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:49.324255 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:49.324367 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:49.324672 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:49.324675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:49.324916 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:49.325047 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:49.325053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:49.326060 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:49.326064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:49.326721 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:49.326732 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:49.327240 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:49.327243 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:49.327311 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:49.327961 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:49.327974 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:49.330516 (+ 2542us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":129,"threads_started":1}
W20260806 16:17:49.330916 25767 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:49.323922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:52381 (local address 127.11.245.190:43267)
0806 16:17:49.324319 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:49.324324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:49.324342 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:17:49.324469 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:49.324472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:49.324530 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:49.324633 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:49.325215 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:49.325902 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:49.326851 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:49.327119 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:49.327213 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:49.330706 (+ 3493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:52381: BlockingRecv error: recv got EOF from 127.11.245.129:52381 (error 108)
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":185,"threads_started":1}
Aug 06 16:17:49 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:49 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:49.565690 25769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:49.558974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:54019)
0806 16:17:49.559374 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:49.559389 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:49.559512 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:49.559814 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:49.559817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:49.560210 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:49.560340 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:49.560347 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:49.561433 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:49.561436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:49.562072 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:49.562083 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:49.562618 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:49.562622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:49.562690 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:49.563349 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:49.563362 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:49.565535 (+ 2173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":169,"threads_started":1}
W20260806 16:17:49.565917 25767 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:49.559084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:54019 (local address 127.11.245.190:43267)
0806 16:17:49.559265 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:49.559270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:49.559494 (+ 224us) server_negotiation.cc:407] Connection header received
0806 16:17:49.559590 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:49.559593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:49.559651 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:49.559735 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:49.560564 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:49.561315 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:49.562205 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:49.562493 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:49.562602 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:49.565753 (+ 3151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:54019: BlockingRecv error: recv got EOF from 127.11.245.130:54019 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 16:17:49 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:49 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:49.816852 25770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:49.809844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:44819)
0806 16:17:49.810202 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:49.810217 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:49.810335 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:49.810605 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:49.810607 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:49.810848 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:49.811024 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:49.811034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:49.812004 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:49.812009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:49.813053 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:49.813069 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:49.813622 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:49.813626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:49.813700 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:49.814394 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:49.814413 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:49.816699 (+ 2286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":134,"threads_started":1}
W20260806 16:17:49.817013 25767 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:49.809932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:44819 (local address 127.11.245.190:43267)
0806 16:17:49.810096 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:49.810101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:49.810290 (+ 189us) server_negotiation.cc:407] Connection header received
0806 16:17:49.810424 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:49.810427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:49.810482 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:49.810561 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:49.811144 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:49.811876 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:49.813201 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:49.813484 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:49.813582 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:49.816868 (+ 3286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:44819: BlockingRecv error: recv got EOF from 127.11.245.131:44819 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 16:17:50 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:50 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:50.338740 25772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:50.331950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:47043)
0806 16:17:50.332415 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:50.332433 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:50.332578 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:50.332886 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:50.332889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:50.333155 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:50.333283 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:50.333289 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:50.334178 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:50.334182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:50.334796 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:50.334804 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:50.335538 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:50.335541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:50.335622 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:50.336391 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:50.336405 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:50.338578 (+ 2173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":173,"threads_started":1}
W20260806 16:17:50.338984 25773 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:50.332165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:47043 (local address 127.11.245.190:43267)
0806 16:17:50.332495 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:50.332499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:50.332517 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:17:50.332677 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:50.332679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:50.332735 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:50.332824 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:50.333433 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:50.334055 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:50.334997 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:50.335377 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:50.335506 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:50.338802 (+ 3296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:47043: BlockingRecv error: recv got EOF from 127.11.245.129:47043 (error 108)
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":126,"threads_started":1}
Aug 06 16:17:50 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:50 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:50.574116 25774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:50.566858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:59855)
0806 16:17:50.567215 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:50.567231 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:50.567376 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:50.567736 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:50.567739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:50.568022 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:50.568236 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:50.568244 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:50.569391 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:50.569397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:50.570329 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:50.570343 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:50.570854 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:50.570857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:50.570938 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:50.571636 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:50.571650 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:50.573952 (+ 2302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":132,"threads_started":1}
W20260806 16:17:50.574317 25773 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:50.567005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:59855 (local address 127.11.245.190:43267)
0806 16:17:50.567180 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:50.567185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:50.567345 (+ 160us) server_negotiation.cc:407] Connection header received
0806 16:17:50.567477 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:50.567480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:50.567558 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:50.567656 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:50.568431 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:50.569219 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:50.570462 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:50.570701 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:50.570791 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:50.574135 (+ 3344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:59855: BlockingRecv error: recv got EOF from 127.11.245.130:59855 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 16:17:50 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:50 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:50.825376 25775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:50.818201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:34507)
0806 16:17:50.818583 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:50.818602 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:50.818769 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:50.819079 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:50.819082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:50.819327 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:50.819465 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:50.819471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:50.820430 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:50.820433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:50.821263 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:50.821278 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:50.821887 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:50.821891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:50.821985 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:50.822699 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:50.822714 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:50.825095 (+ 2381us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":121,"threads_started":1}
W20260806 16:17:50.825424 25773 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:50.818307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:34507 (local address 127.11.245.190:43267)
0806 16:17:50.818473 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:50.818480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:50.818710 (+ 230us) server_negotiation.cc:407] Connection header received
0806 16:17:50.818889 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:50.818891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:50.818953 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:50.819044 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:50.819594 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:50.820284 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:50.821397 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:50.821767 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:50.821882 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:50.825300 (+ 3418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:34507: BlockingRecv error: recv got EOF from 127.11.245.131:34507 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 16:17:51 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:51 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:51.347236 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:51.340029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:34833)
0806 16:17:51.340551 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:51.340575 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:51.340727 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:51.341103 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:51.341107 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:51.341412 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:51.341596 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:51.341606 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:51.342535 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:51.342538 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:51.343132 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:51.343141 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:51.343720 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:51.343723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:51.343842 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:51.344614 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:51.344628 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:51.347077 (+ 2449us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":121,"threads_started":1}
W20260806 16:17:51.347644 25779 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:51.340156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:34833 (local address 127.11.245.190:43267)
0806 16:17:51.340562 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:51.340568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:51.340655 (+ 87us) server_negotiation.cc:407] Connection header received
0806 16:17:51.340850 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:51.340853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:51.340939 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:51.341041 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:51.341734 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:51.342377 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:51.343267 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:51.343576 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:51.343680 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:51.347477 (+ 3797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:34833: BlockingRecv error: recv got EOF from 127.11.245.129:34833 (error 108)
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":109,"threads_started":1}
Aug 06 16:17:51 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:51 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:51.582660 25781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:51.575565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:57837)
0806 16:17:51.576203 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:51.576220 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:51.576392 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:51.576736 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:51.576739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:51.577063 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:51.577223 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:51.577232 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:51.578198 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:51.578201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:51.578820 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:51.578828 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:51.579371 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:51.579374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:51.579451 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:51.580151 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:51.580167 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:51.582502 (+ 2335us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":147,"threads_started":1}
W20260806 16:17:51.582980 25779 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:51.575683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:57837 (local address 127.11.245.190:43267)
0806 16:17:51.575920 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:51.575927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:51.576318 (+ 391us) server_negotiation.cc:407] Connection header received
0806 16:17:51.576510 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:51.576517 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:51.576576 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:51.576674 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:51.577393 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:51.578066 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:51.578984 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:51.579235 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:51.579331 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:51.582820 (+ 3489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:57837: BlockingRecv error: recv got EOF from 127.11.245.130:57837 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 06 16:17:51 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:51 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:51.833276 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:51.826727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:55045)
0806 16:17:51.827185 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:51.827200 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:51.827429 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:51.827759 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:51.827763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:51.828037 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:51.828185 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:51.828192 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:51.829032 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:51.829036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:51.829632 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:51.829640 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:51.830120 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:51.830124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:51.830220 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:51.830880 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:51.830894 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:51.833126 (+ 2232us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":132,"threads_started":1}
W20260806 16:17:51.833463 25779 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:51.826795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:55045 (local address 127.11.245.190:43267)
0806 16:17:51.826995 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:51.827002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:51.827309 (+ 307us) server_negotiation.cc:407] Connection header received
0806 16:17:51.827572 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:51.827575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:51.827636 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:51.827726 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:51.828309 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:51.828901 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:51.829751 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:51.829999 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:51.830111 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:51.833286 (+ 3175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:55045: BlockingRecv error: recv got EOF from 127.11.245.131:55045 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 06 16:17:52 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:52 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:52.354571 25783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:52.348376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:42087)
0806 16:17:52.348740 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:52.348756 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:52.348874 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:52.349154 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:52.349156 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:52.349383 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:52.349509 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:52.349515 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:52.350311 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:52.350314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:52.350905 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:52.350932 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:52.351437 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:52.351440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:52.351515 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:52.352173 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:52.352186 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:52.354414 (+ 2228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":118,"threads_started":1}
W20260806 16:17:52.354842 25787 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:52.348558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:42087 (local address 127.11.245.190:43267)
0806 16:17:52.348827 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:52.348832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:52.348846 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:17:52.348976 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:52.348978 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:52.349029 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:52.349114 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:52.349616 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:52.350207 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:52.351044 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:52.351295 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:52.351407 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:52.354646 (+ 3239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:42087: BlockingRecv error: recv got EOF from 127.11.245.129:42087 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":95,"threads_started":1}
Aug 06 16:17:52 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:52 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:52.596716 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:52.583907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:52713)
0806 16:17:52.584384 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:52.584400 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:52.584891 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:52.585681 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:52.585685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:52.586057 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:52.586209 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:52.586218 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:52.587278 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:52.587281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:52.588005 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:52.588017 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:52.588658 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:52.588661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:52.588734 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:52.594123 (+ 5389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:52.594140 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:52.596543 (+ 2403us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":174,"threads_started":1}
W20260806 16:17:52.596958 25787 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:52.584101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:52713 (local address 127.11.245.190:43267)
0806 16:17:52.584280 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:52.584285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:52.584736 (+ 451us) server_negotiation.cc:407] Connection header received
0806 16:17:52.585098 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:52.585100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:52.585168 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:52.585484 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:52.586385 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:52.587149 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:52.588242 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:52.588509 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:52.588650 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:52.596789 (+ 8139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:52713: BlockingRecv error: recv got EOF from 127.11.245.130:52713 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 16:17:52 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:52 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:52.840641 25789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:52.834394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:40711)
0806 16:17:52.834718 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:52.834733 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:52.834897 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:52.835195 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:52.835198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:52.835459 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:52.835588 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:52.835594 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:52.836509 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:52.836513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:52.837102 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:52.837111 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:52.837597 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:52.837601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:52.837694 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:52.838313 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:52.838325 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:52.840416 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":106,"threads_started":1}
W20260806 16:17:52.840679 25787 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:52.834457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:40711 (local address 127.11.245.190:43267)
0806 16:17:52.834622 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:52.834630 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:52.834821 (+ 191us) server_negotiation.cc:407] Connection header received
0806 16:17:52.835013 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:52.835016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:52.835080 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:52.835159 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:52.835712 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:52.836392 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:52.837226 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:52.837475 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:52.837587 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:52.840562 (+ 2975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:40711: BlockingRecv error: recv got EOF from 127.11.245.131:40711 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 16:17:53 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:53 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:53.362963 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:53.355681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:46265)
0806 16:17:53.356123 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:53.356144 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:53.356283 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:53.356656 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:53.356659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:53.356926 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:53.357059 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:53.357065 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:53.357966 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:53.357970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:53.358678 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:53.358692 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:53.359245 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:53.359248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:53.359324 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:53.360147 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:53.360163 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:53.362795 (+ 2632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":119,"threads_started":1}
W20260806 16:17:53.363159 25791 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:53.355708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:46265 (local address 127.11.245.190:43267)
0806 16:17:53.356124 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:53.356129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:53.356218 (+ 89us) server_negotiation.cc:407] Connection header received
0806 16:17:53.356422 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:53.356424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:53.356483 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:53.356574 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:53.357185 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:53.357821 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:53.358827 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:53.359089 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:53.359191 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:53.362997 (+ 3806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:46265: BlockingRecv error: recv got EOF from 127.11.245.129:46265 (error 108)
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":119,"threads_started":1}
Aug 06 16:17:53 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:53 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:53.606424 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:53.597813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:42267)
0806 16:17:53.598211 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:53.598227 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:53.598365 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:53.598722 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:53.598727 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:53.599019 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:53.599192 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:53.599204 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:53.600277 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:53.600284 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:53.601367 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:53.601381 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:53.601877 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:53.601881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:53.602060 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:53.603418 (+ 1358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:53.603453 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:53.606230 (+ 2777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":186,"threads_started":1}
W20260806 16:17:53.606647 25791 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:53.597907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:42267 (local address 127.11.245.190:43267)
0806 16:17:53.598073 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:53.598077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:53.598303 (+ 226us) server_negotiation.cc:407] Connection header received
0806 16:17:53.598475 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:53.598479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:53.598546 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:53.598657 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:53.599328 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:53.600156 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:53.601498 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:53.601732 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:53.601816 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:53.606471 (+ 4655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:42267: BlockingRecv error: recv got EOF from 127.11.245.130:42267 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 16:17:53 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:53 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 16:17:53.849201 25791 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:53.841972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:39835 (local address 127.11.245.190:43267)
0806 16:17:53.842222 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:53.842229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:53.842393 (+ 164us) server_negotiation.cc:407] Connection header received
0806 16:17:53.842609 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:53.842611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:53.842668 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:53.842756 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:53.843388 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:53.843979 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:53.845030 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:53.845269 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:53.845388 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:53.849057 (+ 3669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:39835: BlockingRecv error: recv got EOF from 127.11.245.131:39835 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
I20260806 16:17:53.849148 25794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:53.841757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:39835)
0806 16:17:53.842281 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:53.842296 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:53.842476 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:53.842791 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:53.842794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:53.843088 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:53.843261 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:53.843268 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:53.844172 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:53.844175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:53.844899 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:53.844911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:53.845404 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:53.845408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:53.845509 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:53.846167 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:53.846180 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:53.848881 (+ 2701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":131,"threads_started":1}
Aug 06 16:17:54 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:54 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:54.371121 25796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:54.364264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:56221)
0806 16:17:54.364682 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:54.364705 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:54.364869 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:54.365214 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:54.365218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:54.365506 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:54.365630 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:54.365637 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:54.366558 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:54.366564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:54.367225 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:54.367234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:54.367772 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:54.367777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:54.367881 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:54.368590 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:54.368603 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:54.370906 (+ 2303us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":139,"threads_started":1}
W20260806 16:17:54.371294 25797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:54.364283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:56221 (local address 127.11.245.190:43267)
0806 16:17:54.364640 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:54.364647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:54.364794 (+ 147us) server_negotiation.cc:407] Connection header received
0806 16:17:54.364997 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:54.365000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:54.365066 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:54.365183 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:54.365767 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:54.366426 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:54.367360 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:54.367636 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:54.367761 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:54.371121 (+ 3360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:56221: BlockingRecv error: recv got EOF from 127.11.245.129:56221 (error 108)
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":129,"threads_started":1}
Aug 06 16:17:54 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:54 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:54.614710 25798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:54.607609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:37933)
0806 16:17:54.608002 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:54.608017 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:54.608204 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:54.608715 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:54.608718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:54.608966 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:54.609094 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:54.609101 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:54.609972 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:54.609976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:54.610665 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:54.610676 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:54.611147 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:54.611150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:54.611224 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:54.611900 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:54.611912 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:54.614470 (+ 2558us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":156,"threads_started":1}
W20260806 16:17:54.614945 25797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:54.607661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:37933 (local address 127.11.245.190:43267)
0806 16:17:54.607852 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:54.607858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:54.608148 (+ 290us) server_negotiation.cc:407] Connection header received
0806 16:17:54.608356 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:54.608359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:54.608538 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:54.608628 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:54.609222 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:54.609834 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:54.610782 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:54.611024 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:54.611109 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:54.614726 (+ 3617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:37933: BlockingRecv error: recv got EOF from 127.11.245.130:37933 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 16:17:54 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:54 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:54.859015 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:54.850749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:40871)
0806 16:17:54.851118 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:54.851141 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:54.851303 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:54.851715 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:54.851719 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:54.852052 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:54.852225 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:54.852233 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:54.853392 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:54.853396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:54.854406 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:54.854422 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:54.855135 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:54.855140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:54.855253 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:54.856128 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:54.856149 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:54.858760 (+ 2611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":124,"threads_started":1}
W20260806 16:17:54.859059 25797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:54.850962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:40871 (local address 127.11.245.190:43267)
0806 16:17:54.851128 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:54.851133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:54.851233 (+ 100us) server_negotiation.cc:407] Connection header received
0806 16:17:54.851426 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:54.851432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:54.851539 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:54.851652 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:54.852406 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:54.853227 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:54.854603 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:54.854974 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:54.855127 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:54.858933 (+ 3806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:40871: BlockingRecv error: recv got EOF from 127.11.245.131:40871 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 16:17:55 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:55 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:55.379882 25800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:55.372551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:58675)
0806 16:17:55.372921 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:55.372939 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:55.373129 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:55.373570 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:55.373573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:55.373877 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:55.374068 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:55.374077 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:55.375219 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:55.375225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:55.375871 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:55.375882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:55.376498 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:55.376502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:55.376608 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:55.377294 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:55.377308 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:55.379694 (+ 2386us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":124,"threads_started":1}
W20260806 16:17:55.380051 25801 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:55.372650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:58675 (local address 127.11.245.190:43267)
0806 16:17:55.372975 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:55.372979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:55.373001 (+ 22us) server_negotiation.cc:407] Connection header received
0806 16:17:55.373295 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:55.373299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:55.373374 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:55.373521 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:55.374238 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:55.375072 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:55.375995 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:55.376342 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:55.376440 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:55.379889 (+ 3449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:58675: BlockingRecv error: recv got EOF from 127.11.245.129:58675 (error 108)
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":141,"threads_started":1}
Aug 06 16:17:55 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:55 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:55.622751 25804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:55.616014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:49501)
0806 16:17:55.616555 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:55.616571 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:55.616715 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:55.617075 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:55.617078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:55.617357 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:55.617488 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:55.617496 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:55.618376 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:55.618379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:55.618988 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:55.618997 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:55.619513 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:55.619517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:55.619604 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:55.620259 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:55.620273 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:55.622580 (+ 2307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":224,"threads_started":1}
W20260806 16:17:55.622978 25801 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:55.616220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:49501 (local address 127.11.245.190:43267)
0806 16:17:55.616455 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:55.616462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:55.616682 (+ 220us) server_negotiation.cc:407] Connection header received
0806 16:17:55.616831 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:55.616834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:55.616892 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:55.616979 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:55.617638 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:55.618237 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:55.619114 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:55.619343 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:55.619438 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:55.622802 (+ 3364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:49501: BlockingRecv error: recv got EOF from 127.11.245.130:49501 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 06 16:17:55 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:55 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:55.868637 25805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:55.860211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:55567)
0806 16:17:55.860656 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:55.860671 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:55.860805 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:55.861301 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:55.861305 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:55.861553 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:55.861683 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:55.861689 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:55.862587 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:55.862591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:55.863371 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:55.863390 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:55.863998 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:55.864003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:55.864155 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:55.864857 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:55.864871 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:55.868459 (+ 3588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":136,"threads_started":1}
W20260806 16:17:55.868794 25801 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:55.860399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:55567 (local address 127.11.245.190:43267)
0806 16:17:55.860657 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:55.860663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:55.860753 (+ 90us) server_negotiation.cc:407] Connection header received
0806 16:17:55.861092 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:55.861095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:55.861156 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:55.861246 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:55.861798 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:55.862392 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:55.863529 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:55.863833 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:55.863947 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:55.868646 (+ 4699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:55567: BlockingRecv error: recv got EOF from 127.11.245.131:55567 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
Aug 06 16:17:56 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:56 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:56.389256 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:56.382019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:37259)
0806 16:17:56.382476 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:56.382501 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:56.382655 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:56.382967 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:56.382970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:56.383214 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:56.383348 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:56.383355 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:56.384852 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:56.384857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:56.385466 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:56.385478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:56.386007 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:56.386011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:56.386107 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:56.386728 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:56.386741 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:56.389088 (+ 2347us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":122,"threads_started":1}
W20260806 16:17:56.389510 25808 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:56.382042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:37259 (local address 127.11.245.190:43267)
0806 16:17:56.382503 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:56.382509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:56.382636 (+ 127us) server_negotiation.cc:407] Connection header received
0806 16:17:56.382739 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:56.382742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:56.382800 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:56.382907 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:56.383695 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:56.384721 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:56.385603 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:56.385866 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:56.385980 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:56.389280 (+ 3300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:37259: BlockingRecv error: recv got EOF from 127.11.245.129:37259 (error 108)
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":125,"threads_started":1}
Aug 06 16:17:56 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:56 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:56.631500 25809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:56.624349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:41857)
0806 16:17:56.624734 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:56.624751 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:56.624894 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:56.625226 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:56.625228 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:56.625464 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:56.625607 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:56.625614 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:56.626514 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:56.626519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:56.627282 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:56.627295 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:56.627907 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:56.627911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:56.628005 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:56.628879 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:56.628899 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:56.631340 (+ 2441us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":138,"threads_started":1}
W20260806 16:17:56.631745 25808 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:56.624426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:41857 (local address 127.11.245.190:43267)
0806 16:17:56.624624 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:56.624630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:56.624845 (+ 215us) server_negotiation.cc:407] Connection header received
0806 16:17:56.625031 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:56.625034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:56.625106 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:56.625191 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:56.625754 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:56.626376 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:56.627437 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:56.627749 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:56.627889 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:56.631585 (+ 3696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:41857: BlockingRecv error: recv got EOF from 127.11.245.130:41857 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 16:17:56 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:56 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:56.876538 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:56.869792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:33939)
0806 16:17:56.870164 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:56.870179 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:56.870316 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:56.870596 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:56.870598 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:56.870842 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:56.870981 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:56.870988 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:56.871780 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:56.871783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:56.872524 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:56.872537 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:56.873164 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:56.873170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:56.873265 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:56.873984 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:56.873997 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:56.876385 (+ 2388us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":138,"threads_started":1}
W20260806 16:17:56.876730 25808 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:56.869854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:33939 (local address 127.11.245.190:43267)
0806 16:17:56.870052 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:56.870059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:56.870267 (+ 208us) server_negotiation.cc:407] Connection header received
0806 16:17:56.870411 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:56.870413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:56.870476 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:56.870551 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:56.871100 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:56.871683 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:56.872683 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:56.873015 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:56.873134 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:56.876554 (+ 3420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:33939: BlockingRecv error: recv got EOF from 127.11.245.131:33939 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 06 16:17:57 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:57 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:57.397308 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:57.390358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:57525)
0806 16:17:57.390818 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:57.390834 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:57.390971 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:57.391235 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:57.391237 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:57.391536 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:57.391668 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:57.391674 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:57.392624 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:57.392628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:57.393257 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:57.393267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:57.393742 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:57.393746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:57.393826 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:57.394632 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:57.394648 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:57.397147 (+ 2499us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":127,"threads_started":1}
W20260806 16:17:57.397644 25812 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:57.390431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:57525 (local address 127.11.245.190:43267)
0806 16:17:57.390775 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:57.390780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:57.390915 (+ 135us) server_negotiation.cc:407] Connection header received
0806 16:17:57.391054 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:57.391056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:57.391109 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:57.391193 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:57.391796 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:57.392492 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:57.393364 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:57.393625 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:57.393707 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:57.397448 (+ 3741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:57525: BlockingRecv error: recv got EOF from 127.11.245.129:57525 (error 108)
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":116,"threads_started":1}
Aug 06 16:17:57 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:57 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:57.643332 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:57.632765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:32991)
0806 16:17:57.633165 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:57.633180 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:57.633321 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:57.633678 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:57.633684 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:57.633937 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:57.634087 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:57.634098 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:57.635208 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:57.635214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:57.635864 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:57.635874 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:57.636491 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:57.636494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:57.636574 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:57.640574 (+ 4000us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:57.640592 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:57.643143 (+ 2551us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":131,"threads_started":1}
W20260806 16:17:57.643597 25812 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:57.632955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:32991 (local address 127.11.245.190:43267)
0806 16:17:57.633175 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:57.633182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:57.633268 (+ 86us) server_negotiation.cc:407] Connection header received
0806 16:17:57.633431 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:57.633435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:57.633518 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:57.633611 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:57.634221 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:57.635092 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:57.636004 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:57.636349 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:57.636442 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:57.643361 (+ 6919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:32991: BlockingRecv error: recv got EOF from 127.11.245.130:32991 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 06 16:17:57 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:57 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:57.885131 25814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:57.877638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:52417)
0806 16:17:57.878038 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:57.878061 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:57.878242 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:57.878596 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:57.878600 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:57.878883 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:57.879010 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:57.879017 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:57.880036 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:57.880043 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:57.880990 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:57.881002 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:57.881571 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:57.881575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:57.881655 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:57.882353 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:57.882367 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:57.884875 (+ 2508us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":122,"threads_started":1}
W20260806 16:17:57.885181 25812 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:57.877733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:52417 (local address 127.11.245.190:43267)
0806 16:17:57.877957 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:57.877963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:57.878160 (+ 197us) server_negotiation.cc:407] Connection header received
0806 16:17:57.878366 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:57.878370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:57.878437 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:57.878543 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:57.879150 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:57.879903 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:57.881162 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:57.881425 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:57.881539 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:57.885047 (+ 3508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:52417: BlockingRecv error: recv got EOF from 127.11.245.131:52417 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 06 16:17:58 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:58 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:58.405911 25815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:58.398457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:43901)
0806 16:17:58.398862 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:58.398879 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:58.399009 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:58.399301 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:58.399304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:58.399554 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:58.399682 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:58.399689 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:58.400859 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:58.400863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:58.401744 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:58.401755 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:58.402414 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:58.402418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:58.402511 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:58.403179 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:58.403193 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:58.405736 (+ 2543us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":137,"threads_started":1}
W20260806 16:17:58.406059 25816 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:58.398522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:43901 (local address 127.11.245.190:43267)
0806 16:17:58.398946 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:58.398951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:58.398968 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:17:58.399111 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:58.399113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:58.399169 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:58.399266 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:58.399818 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:58.400711 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:58.401888 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:58.402247 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:58.402404 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:58.405894 (+ 3490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:43901: BlockingRecv error: recv got EOF from 127.11.245.129:43901 (error 108)
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":106,"threads_started":1}
Aug 06 16:17:58 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:58 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:58.651787 25819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:58.644770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:34405)
0806 16:17:58.645129 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:58.645144 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:58.645279 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:58.645585 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:58.645588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:58.645832 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:58.645954 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:58.645960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:58.647038 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:58.647042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:58.647731 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:58.647742 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:58.648260 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:58.648263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:58.648347 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:58.649001 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:58.649015 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:58.651610 (+ 2595us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":131,"threads_started":1}
W20260806 16:17:58.651981 25816 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:58.644840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:34405 (local address 127.11.245.190:43267)
0806 16:17:58.645012 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:58.645017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:58.645289 (+ 272us) server_negotiation.cc:407] Connection header received
0806 16:17:58.645383 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:58.645386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:58.645446 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:58.645531 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:58.646259 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:58.646899 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:58.647872 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:58.648146 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:58.648243 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:58.651798 (+ 3555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:34405: BlockingRecv error: recv got EOF from 127.11.245.130:34405 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 16:17:58 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:58 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.131@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:58.893309 25820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:58.886182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.131:32931)
0806 16:17:58.886587 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:58.886602 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:58.886765 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:58.887056 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:58.887060 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:58.887342 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:58.887641 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:58.887651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:58.888715 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:58.888719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:58.889392 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:58.889402 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:58.889893 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:58.889896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:58.890000 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:58.890714 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:58.890728 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:58.893067 (+ 2339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":165,"threads_started":1}
W20260806 16:17:58.893378 25816 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:58.886255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:32931 (local address 127.11.245.190:43267)
0806 16:17:58.886432 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:58.886438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:58.886687 (+ 249us) server_negotiation.cc:407] Connection header received
0806 16:17:58.886876 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:58.886880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:58.886943 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:58.887029 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:58.887803 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:58.888559 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:58.889528 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:58.889764 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:58.889880 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:58.893233 (+ 3353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.131:32931: BlockingRecv error: recv got EOF from 127.11.245.131:32931 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 16:17:59 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:59 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.129@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:59.413965 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:59.407078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.129:37931)
0806 16:17:59.407500 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:59.407525 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:59.407654 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:59.407998 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:59.408001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:59.408300 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:59.408425 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:59.408433 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:59.409346 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:59.409348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:59.409954 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:59.409963 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:59.410479 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:59.410482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:59.410555 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:59.411355 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:59.411370 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:59.413757 (+ 2387us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":127,"threads_started":1}
W20260806 16:17:59.414145 25822 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:59.407142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:37931 (local address 127.11.245.190:43267)
0806 16:17:59.407478 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:59.407482 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:59.407610 (+ 128us) server_negotiation.cc:407] Connection header received
0806 16:17:59.407790 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:59.407792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:59.407850 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:59.407950 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:59.408596 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:59.409236 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:59.410081 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:59.410329 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:59.410429 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:59.413965 (+ 3536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.129:37931: BlockingRecv error: recv got EOF from 127.11.245.129:37931 (error 108)
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":112,"threads_started":1}
Aug 06 16:17:59 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:17:59 dist-test-slave-t3q3 krb5kdc[24310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.245.130@KRBTEST.COM for oryx/127.11.245.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:17:59.660939 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:17:59.653435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43267 (local address 127.11.245.130:44661)
0806 16:17:59.653797 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 16:17:59.653819 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:17:59.653965 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:17:59.654426 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:17:59.654430 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:17:59.654713 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:17:59.654853 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:59.654861 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:59.655796 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:59.655800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:59.656514 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:17:59.656527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:17:59.657036 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:17:59.657040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:17:59.657131 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:59.657806 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:17:59.657821 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:17:59.660778 (+ 2957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.245.190:43267: Server oryx/127.11.245.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":142,"threads_started":1}
W20260806 16:17:59.661286 25822 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:17:59.653501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:44661 (local address 127.11.245.190:43267)
0806 16:17:59.653701 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 16:17:59.653708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:17:59.653990 (+ 282us) server_negotiation.cc:407] Connection header received
0806 16:17:59.654053 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:17:59.654055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:17:59.654152 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:17:59.654253 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:17:59.655014 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:59.655644 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:59.656650 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:17:59.656893 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:17:59.656985 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:17:59.661070 (+ 4085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.245.130:44661: BlockingRecv error: recv got EOF from 127.11.245.130:44661 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 16:17:59.758615 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 25153
I20260806 16:17:59.787350 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 25370
I20260806 16:17:59.810292 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 25558
I20260806 16:17:59.829031 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 25061
2026-08-06T16:17:59Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18936 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T16:17:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:17:59Z Disabled control of system clock
I20260806 16:17:59.876703 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:39895
--webserver_interface=127.11.245.190
--webserver_port=0
--builtin_ntp_servers=127.11.245.148:40699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:39895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 16:18:00.022230 25831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:18:00.022629 25831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:18:00.022670 25831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:18:00.027544 25831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:18:00.027666 25831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:18:00.027683 25831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:18:00.027702 25831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:18:00.027720 25831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:18:00.033406 25831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:39895
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:39895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:18:00.035094 25831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:18:00.036948 25831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:18:00.048156 25842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:00.048156 25840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:00.048254 25831 server_base.cc:1034] running on GCE node
W20260806 16:18:00.048194 25839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:00.049207 25831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:18:00.050686 25831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:18:00.051882 25831 hybrid_clock.cc:648] HybridClock initialized: now 1786033080051858 us; error 34 us; skew 500 ppm
I20260806 16:18:00.054865 25831 webserver.cc:460] Webserver started at http://127.11.245.190:46631/ using document root <none> and password file <none>
I20260806 16:18:00.055594 25831 fs_manager.cc:359] Metadata directory not provided
I20260806 16:18:00.055665 25831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:18:00.055886 25831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:18:00.058616 25831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "67305efd69994bdda81d4a4af2d6f034"
format_stamp: "Formatted at 2026-08-06 16:18:00 on dist-test-slave-t3q3"
server_key: "fa698e86fc00817a2e3e309921a96a94"
server_key_iv: "ded20a8ed0739f3b0e40718bfd801ed5"
server_key_version: "encryptionkey@0"
I20260806 16:18:00.059264 25831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "67305efd69994bdda81d4a4af2d6f034"
format_stamp: "Formatted at 2026-08-06 16:18:00 on dist-test-slave-t3q3"
server_key: "fa698e86fc00817a2e3e309921a96a94"
server_key_iv: "ded20a8ed0739f3b0e40718bfd801ed5"
server_key_version: "encryptionkey@0"
I20260806 16:18:00.063992 25831 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 16:18:00.067557 25855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:00.069200 25831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:18:00.069367 25831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "67305efd69994bdda81d4a4af2d6f034"
format_stamp: "Formatted at 2026-08-06 16:18:00 on dist-test-slave-t3q3"
server_key: "fa698e86fc00817a2e3e309921a96a94"
server_key_iv: "ded20a8ed0739f3b0e40718bfd801ed5"
server_key_version: "encryptionkey@0"
I20260806 16:18:00.069527 25831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:18:00.088624 25831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:18:00.089649 25831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:18:00.089936 25831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:18:00.098933 25831 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:39895
I20260806 16:18:00.098949 25925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:39895 every 8 connection(s)
I20260806 16:18:00.100281 25831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:18:00.104977 25926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:18:00.105949 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 25831
I20260806 16:18:00.106110 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:18:00.106523 12246 external_mini_cluster.cc:1428] Setting key d043a4acd62aab5004141ab30b8340be
I20260806 16:18:00.113118 25926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034: Bootstrap starting.
I20260806 16:18:00.116401 25926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034: Neither blocks nor log segments found. Creating new log.
I20260806 16:18:00.117424 25926 log.cc:824] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034: Log is configured to *not* fsync() on all Append() calls
I20260806 16:18:00.118734 25929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:00.108618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36646 (local address 127.11.245.190:39895)
0806 16:18:00.109510 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:00.109541 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:00.109590 (+ 49us) server_negotiation.cc:407] Connection header received
0806 16:18:00.110385 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:00.110407 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:00.110908 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:00.111358 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:18:00.111922 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:00.113068 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:00.114093 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:00.114609 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:00.114796 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:00.116034 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:18:00.116040 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:18:00.116111 (+ 71us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:18:00.116669 (+ 558us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:18:00.116754 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:18:00.116780 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:18:00.117040 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:00.117200 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 16:18:00.117592 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":192,"threads_started":1}
I20260806 16:18:00.120450 25926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034: No bootstrap required, opened a new log
I20260806 16:18:00.124996 25926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67305efd69994bdda81d4a4af2d6f034" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39895 } }
I20260806 16:18:00.125342 25926 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:18:00.125450 25926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67305efd69994bdda81d4a4af2d6f034, State: Initialized, Role: FOLLOWER
I20260806 16:18:00.126048 25926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67305efd69994bdda81d4a4af2d6f034" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39895 } }
I20260806 16:18:00.126230 25926 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:18:00.126327 25926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:18:00.126452 25926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:18:00.127703 25926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67305efd69994bdda81d4a4af2d6f034" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39895 } }
I20260806 16:18:00.128201 25926 leader_election.cc:302] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67305efd69994bdda81d4a4af2d6f034; no voters:
I20260806 16:18:00.128641 25926 leader_election.cc:288] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:18:00.128777 25932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:18:00.129052 25932 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [term 1 LEADER]: Becoming Leader. State: Replica: 67305efd69994bdda81d4a4af2d6f034, State: Running, Role: LEADER
I20260806 16:18:00.129467 25932 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67305efd69994bdda81d4a4af2d6f034" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39895 } }
I20260806 16:18:00.130262 25926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:18:00.131590 25935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67305efd69994bdda81d4a4af2d6f034. Latest consensus state: current_term: 1 leader_uuid: "67305efd69994bdda81d4a4af2d6f034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67305efd69994bdda81d4a4af2d6f034" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39895 } } }
I20260806 16:18:00.131744 25935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [sys.catalog]: This master's current role is: LEADER
I20260806 16:18:00.132053 25933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67305efd69994bdda81d4a4af2d6f034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67305efd69994bdda81d4a4af2d6f034" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39895 } } }
I20260806 16:18:00.132184 25933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034 [sys.catalog]: This master's current role is: LEADER
I20260806 16:18:00.132661 25945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:18:00.137147 25945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:18:00.145206 25945 catalog_manager.cc:1347] Generated new cluster ID: 6cc00003c2034b188dada81175720222
I20260806 16:18:00.145324 25945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:18:00.159673 25945 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:18:00.161345 25945 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:18:00.169195 25945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67305efd69994bdda81d4a4af2d6f034: Generated new TSK 0
I20260806 16:18:00.170342 25945 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:18:00.178963 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:39895
--builtin_ntp_servers=127.11.245.148:40699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 16:18:00.334009 25959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:18:00.334378 25959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:18:00.334453 25959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:18:00.338858 25959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:18:00.338937 25959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:18:00.338999 25959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:18:00.344305 25959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:39895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:18:00.345837 25959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:18:00.347705 25959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:18:00.361485 25965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:00.361658 25959 server_base.cc:1034] running on GCE node
W20260806 16:18:00.361485 25967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:00.361919 25964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:00.362498 25959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:18:00.363337 25959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:18:00.364516 25959 hybrid_clock.cc:648] HybridClock initialized: now 1786033080364480 us; error 53 us; skew 500 ppm
I20260806 16:18:00.366998 25959 webserver.cc:460] Webserver started at http://127.11.245.129:37501/ using document root <none> and password file <none>
I20260806 16:18:00.367794 25959 fs_manager.cc:359] Metadata directory not provided
I20260806 16:18:00.368000 25959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:18:00.368341 25959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:18:00.371101 25959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "904a95c6c7eb4da4abc518b287a64383"
format_stamp: "Formatted at 2026-08-06 16:18:00 on dist-test-slave-t3q3"
server_key: "5045cbac4291373fd159b017cdb88c83"
server_key_iv: "86245db99dcb77edea26449574bbea26"
server_key_version: "encryptionkey@0"
I20260806 16:18:00.371804 25959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "904a95c6c7eb4da4abc518b287a64383"
format_stamp: "Formatted at 2026-08-06 16:18:00 on dist-test-slave-t3q3"
server_key: "5045cbac4291373fd159b017cdb88c83"
server_key_iv: "86245db99dcb77edea26449574bbea26"
server_key_version: "encryptionkey@0"
I20260806 16:18:00.377132 25959 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 16:18:00.380957 25975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:00.382714 25959 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:18:00.382906 25959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "904a95c6c7eb4da4abc518b287a64383"
format_stamp: "Formatted at 2026-08-06 16:18:00 on dist-test-slave-t3q3"
server_key: "5045cbac4291373fd159b017cdb88c83"
server_key_iv: "86245db99dcb77edea26449574bbea26"
server_key_version: "encryptionkey@0"
I20260806 16:18:00.383078 25959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:18:00.407572 25959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:18:00.408695 25959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:18:00.408947 25959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:18:00.409787 25959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:18:00.411106 25959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:18:00.411185 25959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:00.411273 25959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:18:00.411325 25959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:00.422454 25959 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:34713
I20260806 16:18:00.422487 26138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:34713 every 8 connection(s)
I20260806 16:18:00.423712 25959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:18:00.430718 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 25959
I20260806 16:18:00.430919 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:18:00.431392 12246 external_mini_cluster.cc:1428] Setting key 7a6fe18668bb1d15fb739a3de792a6a9
I20260806 16:18:00.434644 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:39895
--builtin_ntp_servers=127.11.245.148:40699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 16:18:00.437901 25929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:00.426377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:35713 (local address 127.11.245.190:39895)
0806 16:18:00.426572 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:00.426577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:00.428119 (+ 1542us) server_negotiation.cc:407] Connection header received
0806 16:18:00.429868 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:00.429872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:00.429950 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:00.430042 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:18:00.431502 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:00.432448 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:00.434446 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:00.434863 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:00.435009 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:00.437104 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:18:00.437111 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:18:00.437116 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:18:00.437144 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:18:00.437181 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:18:00.437196 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:18:00.437309 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:00.437528 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 16:18:00.437651 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 16:18:00.439070 26144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:00.426855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:39895 (local address 127.11.245.129:35713)
0806 16:18:00.427854 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0806 16:18:00.427908 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 16:18:00.429575 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:18:00.430325 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:18:00.430334 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:18:00.430441 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:18:00.431286 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:00.431318 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:00.432644 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:00.432648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:00.434261 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:00.434279 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:00.435252 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:00.435258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:00.435588 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:00.436571 (+ 983us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:18:00.436576 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:18:00.436661 (+ 85us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:18:00.436800 (+ 139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:18:00.437334 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:18:00.437343 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:18:00.437350 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 16:18:00.437492 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 16:18:00.437861 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":155,"threads_started":1}
I20260806 16:18:00.440907 26141 heartbeater.cc:347] Connected to a master server at 127.11.245.190:39895
I20260806 16:18:00.441265 26141 heartbeater.cc:464] Registering TS with master...
I20260806 16:18:00.442081 26141 heartbeater.cc:511] Master 127.11.245.190:39895 requested a full tablet report, sending...
I20260806 16:18:00.444320 25878 ts_manager.cc:194] Registered new tserver with Master: 904a95c6c7eb4da4abc518b287a64383 (127.11.245.129:34713)
I20260806 16:18:00.445791 25878 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.245.129:35713
W20260806 16:18:00.587909 26145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:18:00.588478 26145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:18:00.588564 26145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:18:00.593142 26145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:18:00.593246 26145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:18:00.593348 26145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:18:00.598795 26145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:39895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:18:00.600459 26145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:18:00.602399 26145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:18:00.616416 26154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:00.616556 26145 server_base.cc:1034] running on GCE node
W20260806 16:18:00.616417 26152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:00.616425 26157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:00.617347 26145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:18:00.618389 26145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:18:00.619571 26145 hybrid_clock.cc:648] HybridClock initialized: now 1786033080619523 us; error 81 us; skew 500 ppm
I20260806 16:18:00.622179 26145 webserver.cc:460] Webserver started at http://127.11.245.130:43467/ using document root <none> and password file <none>
I20260806 16:18:00.622999 26145 fs_manager.cc:359] Metadata directory not provided
I20260806 16:18:00.623076 26145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:18:00.623363 26145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:18:00.626219 26145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "5bfb739f244a4864b8b5053767727ce3"
format_stamp: "Formatted at 2026-08-06 16:18:00 on dist-test-slave-t3q3"
server_key: "61edc4ee130981652a8272b5c2ee4ab6"
server_key_iv: "11bfa01311dfa031d932b0c9700b2b4e"
server_key_version: "encryptionkey@0"
I20260806 16:18:00.626982 26145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "5bfb739f244a4864b8b5053767727ce3"
format_stamp: "Formatted at 2026-08-06 16:18:00 on dist-test-slave-t3q3"
server_key: "61edc4ee130981652a8272b5c2ee4ab6"
server_key_iv: "11bfa01311dfa031d932b0c9700b2b4e"
server_key_version: "encryptionkey@0"
I20260806 16:18:00.632397 26145 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 16:18:00.635743 26166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:00.637633 26145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:18:00.637812 26145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "5bfb739f244a4864b8b5053767727ce3"
format_stamp: "Formatted at 2026-08-06 16:18:00 on dist-test-slave-t3q3"
server_key: "61edc4ee130981652a8272b5c2ee4ab6"
server_key_iv: "11bfa01311dfa031d932b0c9700b2b4e"
server_key_version: "encryptionkey@0"
I20260806 16:18:00.637976 26145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:18:00.659111 26145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:18:00.660257 26145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:18:00.660523 26145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:18:00.661371 26145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:18:00.662739 26145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:18:00.662794 26145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:00.662843 26145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:18:00.662859 26145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:00.676704 26145 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:34099
I20260806 16:18:00.676752 26325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:34099 every 8 connection(s)
I20260806 16:18:00.678014 26145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:18:00.686990 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 26145
I20260806 16:18:00.687196 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:18:00.687620 12246 external_mini_cluster.cc:1428] Setting key 4bc7eec43923ab4f00a8589fe8c4609c
I20260806 16:18:00.691601 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:39895
--builtin_ntp_servers=127.11.245.148:40699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 16:18:00.693522 25929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:00.680738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:55607 (local address 127.11.245.190:39895)
0806 16:18:00.680950 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:00.680956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:00.683048 (+ 2092us) server_negotiation.cc:407] Connection header received
0806 16:18:00.684887 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:00.684891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:00.684961 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:00.685073 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:18:00.686341 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:00.687011 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:00.689756 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:00.690376 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:00.690501 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:00.692647 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:18:00.692655 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:18:00.692662 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:18:00.692689 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:18:00.692727 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:18:00.692742 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:18:00.692849 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:00.693147 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 16:18:00.693264 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 16:18:00.694859 26330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:00.681348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:39895 (local address 127.11.245.130:55607)
0806 16:18:00.682786 (+ 1438us) negotiation.cc:107] Waiting for socket to connect
0806 16:18:00.682844 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 16:18:00.684598 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:18:00.685294 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:18:00.685303 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:18:00.685418 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:18:00.686144 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:00.686160 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:00.687187 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:00.687192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:00.689395 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:00.689432 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:00.690652 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:00.690659 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:00.690968 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:00.692196 (+ 1228us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:18:00.692199 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:18:00.692277 (+ 78us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:18:00.692404 (+ 127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:18:00.692909 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:18:00.692920 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:18:00.692929 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 16:18:00.693107 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 16:18:00.693517 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":179,"threads_started":1}
I20260806 16:18:00.697307 26327 heartbeater.cc:347] Connected to a master server at 127.11.245.190:39895
I20260806 16:18:00.697722 26327 heartbeater.cc:464] Registering TS with master...
I20260806 16:18:00.698602 26327 heartbeater.cc:511] Master 127.11.245.190:39895 requested a full tablet report, sending...
I20260806 16:18:00.700507 25878 ts_manager.cc:194] Registered new tserver with Master: 5bfb739f244a4864b8b5053767727ce3 (127.11.245.130:34099)
I20260806 16:18:00.701633 25878 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.245.130:55607
W20260806 16:18:00.855026 26331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:18:00.855432 26331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:18:00.855477 26331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:18:00.859884 26331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:18:00.859970 26331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:18:00.860040 26331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:18:00.866070 26331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:40699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:39895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:18:00.867717 26331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:18:00.870044 26331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:18:00.883932 26343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:00.884110 26340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:00.884141 26338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:00.884364 26331 server_base.cc:1034] running on GCE node
I20260806 16:18:00.884944 26331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:18:00.885807 26331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:18:00.887005 26331 hybrid_clock.cc:648] HybridClock initialized: now 1786033080886975 us; error 38 us; skew 500 ppm
I20260806 16:18:00.889807 26331 webserver.cc:460] Webserver started at http://127.11.245.131:38895/ using document root <none> and password file <none>
I20260806 16:18:00.890630 26331 fs_manager.cc:359] Metadata directory not provided
I20260806 16:18:00.890705 26331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:18:00.890992 26331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:18:00.894047 26331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "e1f0bdf9bd5145169bb90d5757028d6d"
format_stamp: "Formatted at 2026-08-06 16:18:00 on dist-test-slave-t3q3"
server_key: "3bfcb3b150b5ae1fea43c5f1fe345e0a"
server_key_iv: "df2e149475f6f98b9d903a86f0a590ed"
server_key_version: "encryptionkey@0"
I20260806 16:18:00.894809 26331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "e1f0bdf9bd5145169bb90d5757028d6d"
format_stamp: "Formatted at 2026-08-06 16:18:00 on dist-test-slave-t3q3"
server_key: "3bfcb3b150b5ae1fea43c5f1fe345e0a"
server_key_iv: "df2e149475f6f98b9d903a86f0a590ed"
server_key_version: "encryptionkey@0"
I20260806 16:18:00.899919 26331 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:18:00.904117 26350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:00.906136 26331 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:18:00.906363 26331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "e1f0bdf9bd5145169bb90d5757028d6d"
format_stamp: "Formatted at 2026-08-06 16:18:00 on dist-test-slave-t3q3"
server_key: "3bfcb3b150b5ae1fea43c5f1fe345e0a"
server_key_iv: "df2e149475f6f98b9d903a86f0a590ed"
server_key_version: "encryptionkey@0"
I20260806 16:18:00.906566 26331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:18:00.939853 26331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:18:00.941278 26331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:18:00.941612 26331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:18:00.942551 26331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:18:00.944180 26331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:18:00.944262 26331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:00.944352 26331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:18:00.944403 26331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:00.956398 26331 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:37205
I20260806 16:18:00.956437 26513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:37205 every 8 connection(s)
I20260806 16:18:00.957716 26331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:18:00.965281 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 26331
I20260806 16:18:00.965466 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:18:00.965862 12246 external_mini_cluster.cc:1428] Setting key 11d6999b7a9f8435c069efdbd41e7420
I20260806 16:18:00.970981 25929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:00.960426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:58337 (local address 127.11.245.190:39895)
0806 16:18:00.960639 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:00.960645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:00.962222 (+ 1577us) server_negotiation.cc:407] Connection header received
0806 16:18:00.964326 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:00.964330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:00.964399 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:00.964495 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:18:00.966176 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:00.967025 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:00.968784 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:00.969030 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:00.969122 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:00.970494 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:18:00.970497 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:18:00.970500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:18:00.970514 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:18:00.970536 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:18:00.970545 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:18:00.970608 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:00.970773 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 16:18:00.970836 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 16:18:00.972262 26516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:00.961035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:39895 (local address 127.11.245.131:58337)
0806 16:18:00.962010 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0806 16:18:00.962068 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 16:18:00.963941 (+ 1873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:18:00.964802 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:18:00.964811 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:18:00.964997 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:18:00.965961 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:00.965989 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:00.967203 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:00.967208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:00.968660 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:00.968671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:00.969186 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:00.969189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:00.969359 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:00.970160 (+ 801us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:18:00.970162 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:18:00.970216 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:18:00.970305 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:18:00.970624 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:18:00.970632 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:18:00.970638 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 16:18:00.970755 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 16:18:00.971092 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":183,"threads_started":1}
I20260806 16:18:00.973760 26514 heartbeater.cc:347] Connected to a master server at 127.11.245.190:39895
I20260806 16:18:00.974100 26514 heartbeater.cc:464] Registering TS with master...
I20260806 16:18:00.974885 26514 heartbeater.cc:511] Master 127.11.245.190:39895 requested a full tablet report, sending...
I20260806 16:18:00.976343 25878 ts_manager.cc:194] Registered new tserver with Master: e1f0bdf9bd5145169bb90d5757028d6d (127.11.245.131:37205)
I20260806 16:18:00.977067 25878 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.245.131:58337
I20260806 16:18:00.981324 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 16:18:00.985466 25929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:18:00.984162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36652 (local address 127.11.245.190:39895)
0806 16:18:00.984358 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:00.984365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:00.984502 (+ 137us) server_negotiation.cc:407] Connection header received
0806 16:18:00.984709 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:00.984712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:00.984780 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:00.984893 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:18:00.985261 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36652: BlockingRecv error: recv got EOF from 127.0.0.1:36652 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 16:18:00.985599 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 25959
I20260806 16:18:01.027071 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 26145
I20260806 16:18:01.045248 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 26331
I20260806 16:18:01.074674 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 25831
2026-08-06T16:18:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1244 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T16:18:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:18:01Z Disabled control of system clock
I20260806 16:18:01.122136 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:36487
--webserver_interface=127.11.245.190
--webserver_port=0
--builtin_ntp_servers=127.11.245.148:46611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:36487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 16:18:01.274119 26527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:18:01.274504 26527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:18:01.274549 26527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:18:01.278918 26527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:18:01.279024 26527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:18:01.279042 26527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:18:01.279060 26527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:18:01.279078 26527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:18:01.284622 26527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:46611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:36487
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:36487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:18:01.286348 26527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:18:01.288365 26527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:18:01.300249 26541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:01.300314 26543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:01.300694 26527 server_base.cc:1034] running on GCE node
W20260806 16:18:01.300886 26539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:01.302196 26527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:18:01.305030 26527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:18:01.308473 26527 hybrid_clock.cc:648] HybridClock initialized: now 1786033081308361 us; error 75 us; skew 500 ppm
I20260806 16:18:01.312304 26527 webserver.cc:460] Webserver started at http://127.11.245.190:35225/ using document root <none> and password file <none>
I20260806 16:18:01.313215 26527 fs_manager.cc:359] Metadata directory not provided
I20260806 16:18:01.313297 26527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:18:01.313535 26527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:18:01.316432 26527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "e4a7c1f25d144bebb73e785941bb507d"
format_stamp: "Formatted at 2026-08-06 16:18:01 on dist-test-slave-t3q3"
server_key: "a6db370ebb0672709bef4dc50c6f1a5a"
server_key_iv: "b6cda87504e713998dd23e93c1d2283f"
server_key_version: "encryptionkey@0"
I20260806 16:18:01.317135 26527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "e4a7c1f25d144bebb73e785941bb507d"
format_stamp: "Formatted at 2026-08-06 16:18:01 on dist-test-slave-t3q3"
server_key: "a6db370ebb0672709bef4dc50c6f1a5a"
server_key_iv: "b6cda87504e713998dd23e93c1d2283f"
server_key_version: "encryptionkey@0"
I20260806 16:18:01.322259 26527 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:18:01.325614 26552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:01.327418 26527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 16:18:01.327625 26527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "e4a7c1f25d144bebb73e785941bb507d"
format_stamp: "Formatted at 2026-08-06 16:18:01 on dist-test-slave-t3q3"
server_key: "a6db370ebb0672709bef4dc50c6f1a5a"
server_key_iv: "b6cda87504e713998dd23e93c1d2283f"
server_key_version: "encryptionkey@0"
I20260806 16:18:01.327785 26527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:18:01.333827 26527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:18:01.334664 26527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:18:01.334888 26527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:18:01.343613 26527 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:36487
I20260806 16:18:01.343622 26626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:36487 every 8 connection(s)
I20260806 16:18:01.345146 26527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:18:01.350039 26627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:18:01.351711 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 26527
I20260806 16:18:01.351866 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:18:01.352335 12246 external_mini_cluster.cc:1428] Setting key 8cf11d24912c585ab1c567ef26453070
I20260806 16:18:01.358713 26627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d: Bootstrap starting.
I20260806 16:18:01.359777 26630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:01.354366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60492 (local address 127.11.245.190:36487)
0806 16:18:01.355288 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:01.355308 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:01.355355 (+ 47us) server_negotiation.cc:407] Connection header received
0806 16:18:01.356238 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:01.356263 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:01.356738 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:01.357169 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:18:01.358145 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:18:01.358151 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:18:01.358165 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:18:01.358176 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:18:01.358245 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:18:01.358264 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:18:01.358456 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:01.358537 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 16:18:01.358862 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":186,"threads_started":1}
I20260806 16:18:01.361997 26627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d: Neither blocks nor log segments found. Creating new log.
I20260806 16:18:01.363080 26627 log.cc:824] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d: Log is configured to *not* fsync() on all Append() calls
I20260806 16:18:01.365868 26627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d: No bootstrap required, opened a new log
I20260806 16:18:01.370929 26627 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a7c1f25d144bebb73e785941bb507d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36487 } }
I20260806 16:18:01.371353 26627 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:18:01.371440 26627 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a7c1f25d144bebb73e785941bb507d, State: Initialized, Role: FOLLOWER
I20260806 16:18:01.372187 26627 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a7c1f25d144bebb73e785941bb507d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36487 } }
I20260806 16:18:01.372440 26627 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:18:01.372548 26627 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:18:01.372663 26627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:18:01.374042 26627 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a7c1f25d144bebb73e785941bb507d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36487 } }
I20260806 16:18:01.374485 26627 leader_election.cc:302] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4a7c1f25d144bebb73e785941bb507d; no voters:
I20260806 16:18:01.374900 26627 leader_election.cc:288] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:18:01.375138 26633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:18:01.375430 26633 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [term 1 LEADER]: Becoming Leader. State: Replica: e4a7c1f25d144bebb73e785941bb507d, State: Running, Role: LEADER
I20260806 16:18:01.375842 26633 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a7c1f25d144bebb73e785941bb507d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36487 } }
I20260806 16:18:01.376832 26627 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:18:01.377967 26634 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4a7c1f25d144bebb73e785941bb507d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a7c1f25d144bebb73e785941bb507d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36487 } } }
I20260806 16:18:01.378121 26634 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [sys.catalog]: This master's current role is: LEADER
I20260806 16:18:01.378561 26635 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4a7c1f25d144bebb73e785941bb507d. Latest consensus state: current_term: 1 leader_uuid: "e4a7c1f25d144bebb73e785941bb507d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a7c1f25d144bebb73e785941bb507d" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 36487 } } }
I20260806 16:18:01.378664 26635 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d [sys.catalog]: This master's current role is: LEADER
I20260806 16:18:01.378998 26641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:18:01.383354 26641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:18:01.392140 26641 catalog_manager.cc:1347] Generated new cluster ID: 88585b754b914d57be00e7f5546a07c9
I20260806 16:18:01.392333 26641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:18:01.406688 26641 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:18:01.407275 26641 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:18:01.420755 26641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4a7c1f25d144bebb73e785941bb507d: Generated new TSK 0
I20260806 16:18:01.421818 26641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:18:01.432430 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:36487
--builtin_ntp_servers=127.11.245.148:46611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 16:18:01.587160 26657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:18:01.587531 26657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:18:01.587579 26657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:18:01.592195 26657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:18:01.592339 26657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:18:01.592466 26657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:18:01.598001 26657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:46611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:36487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:18:01.600044 26657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:18:01.602138 26657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:18:01.615181 26665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:01.615278 26667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:01.615171 26664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:01.615818 26657 server_base.cc:1034] running on GCE node
I20260806 16:18:01.616672 26657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:18:01.617950 26657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:18:01.619287 26657 hybrid_clock.cc:648] HybridClock initialized: now 1786033081619270 us; error 62 us; skew 500 ppm
I20260806 16:18:01.622248 26657 webserver.cc:460] Webserver started at http://127.11.245.129:42203/ using document root <none> and password file <none>
I20260806 16:18:01.623169 26657 fs_manager.cc:359] Metadata directory not provided
I20260806 16:18:01.623291 26657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:18:01.623606 26657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:18:01.626549 26657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "d50c73e9f71e491aadee8e8726862979"
format_stamp: "Formatted at 2026-08-06 16:18:01 on dist-test-slave-t3q3"
server_key: "ac973880d8d16e422d975a995f1a987c"
server_key_iv: "d386f870d9049b3724c9bb4f13bbd0b9"
server_key_version: "encryptionkey@0"
I20260806 16:18:01.627300 26657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "d50c73e9f71e491aadee8e8726862979"
format_stamp: "Formatted at 2026-08-06 16:18:01 on dist-test-slave-t3q3"
server_key: "ac973880d8d16e422d975a995f1a987c"
server_key_iv: "d386f870d9049b3724c9bb4f13bbd0b9"
server_key_version: "encryptionkey@0"
I20260806 16:18:01.632361 26657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 16:18:01.635831 26678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:01.637465 26657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:18:01.637640 26657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "d50c73e9f71e491aadee8e8726862979"
format_stamp: "Formatted at 2026-08-06 16:18:01 on dist-test-slave-t3q3"
server_key: "ac973880d8d16e422d975a995f1a987c"
server_key_iv: "d386f870d9049b3724c9bb4f13bbd0b9"
server_key_version: "encryptionkey@0"
I20260806 16:18:01.637794 26657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:18:01.643383 26657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:18:01.644277 26657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:18:01.644490 26657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:18:01.645188 26657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:18:01.646360 26657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:18:01.646436 26657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:01.646521 26657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:18:01.646564 26657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:01.658324 26657 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:40199
I20260806 16:18:01.658345 26851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:40199 every 8 connection(s)
I20260806 16:18:01.659723 26657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:18:01.664458 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 26657
I20260806 16:18:01.664631 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:18:01.665032 12246 external_mini_cluster.cc:1428] Setting key 86bd12aaf2fb446807bd70b37530b256
I20260806 16:18:01.667640 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:36487
--builtin_ntp_servers=127.11.245.148:46611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 16:18:01.668659 26630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:01.662340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:34595 (local address 127.11.245.190:36487)
0806 16:18:01.662531 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:01.662536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:01.664017 (+ 1481us) server_negotiation.cc:407] Connection header received
0806 16:18:01.665977 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:01.665981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:01.666042 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:01.666148 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:18:01.667650 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:18:01.667656 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:18:01.667663 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:18:01.667682 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:18:01.667716 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:18:01.667729 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:18:01.667881 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:01.668255 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 16:18:01.668329 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 16:18:01.669705 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:01.662887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:36487 (local address 127.11.245.129:34595)
0806 16:18:01.663810 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0806 16:18:01.663863 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 16:18:01.665643 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:18:01.666345 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:18:01.666353 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:18:01.666468 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:18:01.667304 (+ 836us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:18:01.667306 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:18:01.667364 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:18:01.667460 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:18:01.668012 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:18:01.668023 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:18:01.668030 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 16:18:01.668288 (+ 258us) client_negotiation.cc:240] Negotiation successful
0806 16:18:01.668574 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":163,"threads_started":1}
I20260806 16:18:01.671311 26852 heartbeater.cc:347] Connected to a master server at 127.11.245.190:36487
I20260806 16:18:01.671698 26852 heartbeater.cc:464] Registering TS with master...
I20260806 16:18:01.672446 26852 heartbeater.cc:511] Master 127.11.245.190:36487 requested a full tablet report, sending...
I20260806 16:18:01.674631 26576 ts_manager.cc:194] Registered new tserver with Master: d50c73e9f71e491aadee8e8726862979 (127.11.245.129:40199)
W20260806 16:18:01.819965 26858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:18:01.820411 26858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:18:01.820506 26858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:18:01.825001 26858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:18:01.825116 26858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:18:01.825218 26858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:18:01.831549 26858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:46611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:36487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:18:01.833874 26858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:18:01.836369 26858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:18:01.850303 26858 server_base.cc:1034] running on GCE node
W20260806 16:18:01.850258 26864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:01.850243 26863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:01.850229 26867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:01.851356 26858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:18:01.852330 26858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:18:01.853525 26858 hybrid_clock.cc:648] HybridClock initialized: now 1786033081853501 us; error 42 us; skew 500 ppm
I20260806 16:18:01.856009 26858 webserver.cc:460] Webserver started at http://127.11.245.130:38045/ using document root <none> and password file <none>
I20260806 16:18:01.856813 26858 fs_manager.cc:359] Metadata directory not provided
I20260806 16:18:01.856887 26858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:18:01.857108 26858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:18:01.859783 26858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "9a6e6037175048ee97838bd0c6903b7f"
format_stamp: "Formatted at 2026-08-06 16:18:01 on dist-test-slave-t3q3"
server_key: "bef8b91624d4e0fb2256f9f8b4f89140"
server_key_iv: "895978d25df6356004365ec2e082dd3b"
server_key_version: "encryptionkey@0"
I20260806 16:18:01.860553 26858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "9a6e6037175048ee97838bd0c6903b7f"
format_stamp: "Formatted at 2026-08-06 16:18:01 on dist-test-slave-t3q3"
server_key: "bef8b91624d4e0fb2256f9f8b4f89140"
server_key_iv: "895978d25df6356004365ec2e082dd3b"
server_key_version: "encryptionkey@0"
I20260806 16:18:01.865602 26858 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 16:18:01.869442 26878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:01.871124 26858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:18:01.871253 26858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "9a6e6037175048ee97838bd0c6903b7f"
format_stamp: "Formatted at 2026-08-06 16:18:01 on dist-test-slave-t3q3"
server_key: "bef8b91624d4e0fb2256f9f8b4f89140"
server_key_iv: "895978d25df6356004365ec2e082dd3b"
server_key_version: "encryptionkey@0"
I20260806 16:18:01.871399 26858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:18:01.877210 26858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:18:01.878207 26858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:18:01.878389 26858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:18:01.879244 26858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:18:01.880548 26858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:18:01.880600 26858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:01.880668 26858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:18:01.880702 26858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:01.891762 26858 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:39345
I20260806 16:18:01.891835 27037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:39345 every 8 connection(s)
I20260806 16:18:01.893323 26858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:18:01.900099 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 26858
I20260806 16:18:01.900290 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:18:01.900748 12246 external_mini_cluster.cc:1428] Setting key 94d2933c0efecad1087cd3d29ed2bb6a
I20260806 16:18:01.901692 26630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:01.895828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:39625 (local address 127.11.245.190:36487)
0806 16:18:01.896028 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:01.896034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:01.897584 (+ 1550us) server_negotiation.cc:407] Connection header received
0806 16:18:01.899295 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:01.899299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:01.899352 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:01.899450 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:18:01.901121 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:18:01.901125 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:18:01.901127 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:18:01.901143 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:18:01.901170 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:18:01.901180 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:18:01.901338 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:01.901487 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 16:18:01.901553 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 16:18:01.903131 27040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:01.896444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:36487 (local address 127.11.245.130:39625)
0806 16:18:01.897371 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0806 16:18:01.897427 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 16:18:01.899027 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:18:01.899743 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:18:01.899753 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:18:01.899891 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:18:01.900720 (+ 829us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:18:01.900722 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:18:01.900790 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:18:01.900899 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:18:01.901314 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:18:01.901323 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:18:01.901329 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 16:18:01.901541 (+ 212us) client_negotiation.cc:240] Negotiation successful
0806 16:18:01.901884 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":154,"threads_started":1}
I20260806 16:18:01.903486 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:36487
--builtin_ntp_servers=127.11.245.148:46611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 16:18:01.904733 27038 heartbeater.cc:347] Connected to a master server at 127.11.245.190:36487
I20260806 16:18:01.905033 27038 heartbeater.cc:464] Registering TS with master...
I20260806 16:18:01.905579 27038 heartbeater.cc:511] Master 127.11.245.190:36487 requested a full tablet report, sending...
I20260806 16:18:01.907161 26576 ts_manager.cc:194] Registered new tserver with Master: 9a6e6037175048ee97838bd0c6903b7f (127.11.245.130:39345)
W20260806 16:18:02.048870 27042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:18:02.049283 27042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:18:02.049368 27042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:18:02.053893 27042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:18:02.054013 27042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:18:02.054118 27042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:18:02.059571 27042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:46611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:36487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:18:02.061296 27042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:18:02.063206 27042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:18:02.076198 27051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:02.076185 27054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:02.076185 27052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:02.076272 27042 server_base.cc:1034] running on GCE node
I20260806 16:18:02.077065 27042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:18:02.078008 27042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:18:02.079226 27042 hybrid_clock.cc:648] HybridClock initialized: now 1786033082079200 us; error 40 us; skew 500 ppm
I20260806 16:18:02.081770 27042 webserver.cc:460] Webserver started at http://127.11.245.131:34519/ using document root <none> and password file <none>
I20260806 16:18:02.082863 27042 fs_manager.cc:359] Metadata directory not provided
I20260806 16:18:02.082954 27042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:18:02.083247 27042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:18:02.086113 27042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "a5e544d943a346c19d7cc03c6cfebb11"
format_stamp: "Formatted at 2026-08-06 16:18:02 on dist-test-slave-t3q3"
server_key: "3b1ab92d6fd39e1b54ed3d345ef65d80"
server_key_iv: "697948ebd547b01b9b79ea5ec166ed8c"
server_key_version: "encryptionkey@0"
I20260806 16:18:02.086839 27042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "a5e544d943a346c19d7cc03c6cfebb11"
format_stamp: "Formatted at 2026-08-06 16:18:02 on dist-test-slave-t3q3"
server_key: "3b1ab92d6fd39e1b54ed3d345ef65d80"
server_key_iv: "697948ebd547b01b9b79ea5ec166ed8c"
server_key_version: "encryptionkey@0"
I20260806 16:18:02.092181 27042 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 16:18:02.096175 27069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:02.097918 27042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:18:02.098109 27042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "a5e544d943a346c19d7cc03c6cfebb11"
format_stamp: "Formatted at 2026-08-06 16:18:02 on dist-test-slave-t3q3"
server_key: "3b1ab92d6fd39e1b54ed3d345ef65d80"
server_key_iv: "697948ebd547b01b9b79ea5ec166ed8c"
server_key_version: "encryptionkey@0"
I20260806 16:18:02.098266 27042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:18:02.103986 27042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:18:02.104823 27042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:18:02.105001 27042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:18:02.105649 27042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:18:02.106820 27042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:18:02.106870 27042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:02.106917 27042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:18:02.106945 27042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:02.118727 27042 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:33623
I20260806 16:18:02.118752 27231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:33623 every 8 connection(s)
I20260806 16:18:02.119971 27042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:18:02.124889 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 27042
I20260806 16:18:02.125051 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:18:02.125468 12246 external_mini_cluster.cc:1428] Setting key 1130930745f9b4317ec7171e74dc77aa
I20260806 16:18:02.129256 26630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:02.122558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:46001 (local address 127.11.245.190:36487)
0806 16:18:02.122754 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:02.122759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:02.124524 (+ 1765us) server_negotiation.cc:407] Connection header received
0806 16:18:02.126693 (+ 2169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:02.126698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:02.126759 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:02.126852 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:18:02.128660 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:18:02.128665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:18:02.128668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:18:02.128690 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:18:02.128714 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:18:02.128724 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:18:02.128853 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:02.129027 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 16:18:02.129088 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 16:18:02.130472 27236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:02.123107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:36487 (local address 127.11.245.131:46001)
0806 16:18:02.124283 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0806 16:18:02.124348 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 16:18:02.126391 (+ 2043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:18:02.127141 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:18:02.127151 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:18:02.127334 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:18:02.128275 (+ 941us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:18:02.128279 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:18:02.128382 (+ 103us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:18:02.128499 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:18:02.128817 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:18:02.128828 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:18:02.128837 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 16:18:02.129091 (+ 254us) client_negotiation.cc:240] Negotiation successful
0806 16:18:02.129437 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":136,"threads_started":1}
I20260806 16:18:02.132126 27234 heartbeater.cc:347] Connected to a master server at 127.11.245.190:36487
I20260806 16:18:02.132476 27234 heartbeater.cc:464] Registering TS with master...
I20260806 16:18:02.133155 27234 heartbeater.cc:511] Master 127.11.245.190:36487 requested a full tablet report, sending...
I20260806 16:18:02.134658 26576 ts_manager.cc:194] Registered new tserver with Master: a5e544d943a346c19d7cc03c6cfebb11 (127.11.245.131:33623)
I20260806 16:18:02.141428 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 16:18:02.145304 26630 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:18:02.144112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60500 (local address 127.11.245.190:36487)
0806 16:18:02.144313 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:02.144319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:02.144496 (+ 177us) server_negotiation.cc:407] Connection header received
0806 16:18:02.144635 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:02.144638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:02.144687 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:02.144769 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:18:02.145103 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60500: BlockingRecv error: recv got EOF from 127.0.0.1:60500 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 16:18:02.145812 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 26657
I20260806 16:18:02.160629 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 26858
I20260806 16:18:02.186076 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 27042
I20260806 16:18:02.216555 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 26527
2026-08-06T16:18:02Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (1146 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:18:02 dist-test-slave-t3q3 krb5kdc[27249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:18:02 dist-test-slave-t3q3 krb5kdc[27249](info): set up 2 sockets
Aug 06 16:18:02 dist-test-slave-t3q3 krb5kdc[27249](info): commencing operation
krb5kdc: starting...
W20260806 16:18:03.509451 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:18:03 dist-test-slave-t3q3 krb5kdc[27249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:18:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:18:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:18:03.676630 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:34485
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:34485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:18:03.829116 27270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:18:03.829473 27270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:18:03.829519 27270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:18:03.833837 27270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:18:03.833927 27270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:18:03.833948 27270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:18:03.833966 27270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:18:03.833984 27270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:18:03.840024 27270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:34485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:34485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:18:03.841660 27270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:18:03.843024 27270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:18:03.855374 27279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:03.855384 27278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:03.855473 27270 server_base.cc:1034] running on GCE node
W20260806 16:18:03.855376 27282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:03.856619 27270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:18:03.858088 27270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:18:03.859261 27270 hybrid_clock.cc:648] HybridClock initialized: now 1786033083859239 us; error 38 us; skew 500 ppm
Aug 06 16:18:03 dist-test-slave-t3q3 krb5kdc[27249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033083, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:18:03.867789 27270 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:18:03.869220 27270 webserver.cc:460] Webserver started at http://127.11.245.190:36823/ using document root <none> and password file <none>
I20260806 16:18:03.869995 27270 fs_manager.cc:359] Metadata directory not provided
I20260806 16:18:03.870070 27270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:18:03.870311 27270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:18:03.873163 27270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "10c40277484f4575be4fe8e5899caad2"
format_stamp: "Formatted at 2026-08-06 16:18:03 on dist-test-slave-t3q3"
server_key: "5c7b995a28c68a220b344defd8fdd442"
server_key_iv: "8b7e2c66211a7ca9695a05f5be1e3fe8"
server_key_version: "encryptionkey@0"
I20260806 16:18:03.873857 27270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "10c40277484f4575be4fe8e5899caad2"
format_stamp: "Formatted at 2026-08-06 16:18:03 on dist-test-slave-t3q3"
server_key: "5c7b995a28c68a220b344defd8fdd442"
server_key_iv: "8b7e2c66211a7ca9695a05f5be1e3fe8"
server_key_version: "encryptionkey@0"
I20260806 16:18:03.879364 27270 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:18:03.883003 27290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:03.885203 27270 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:18:03.885447 27270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "10c40277484f4575be4fe8e5899caad2"
format_stamp: "Formatted at 2026-08-06 16:18:03 on dist-test-slave-t3q3"
server_key: "5c7b995a28c68a220b344defd8fdd442"
server_key_iv: "8b7e2c66211a7ca9695a05f5be1e3fe8"
server_key_version: "encryptionkey@0"
I20260806 16:18:03.885632 27270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:18:03.907509 27270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:18:03.911880 27270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:18:03.912140 27270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:18:03.921475 27270 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:34485
I20260806 16:18:03.921840 27363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:34485 every 8 connection(s)
I20260806 16:18:03.922854 27270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:18:03.927086 27364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:18:03.928401 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 27270
I20260806 16:18:03.928577 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:18:03.928969 12246 external_mini_cluster.cc:1428] Setting key 7651b37002eca008211e67c5f2d7fe68
I20260806 16:18:03.935822 27364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2: Bootstrap starting.
I20260806 16:18:03.939395 27364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2: Neither blocks nor log segments found. Creating new log.
I20260806 16:18:03.940604 27364 log.cc:824] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:18:03 dist-test-slave-t3q3 krb5kdc[27249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:18:03.943728 27364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2: No bootstrap required, opened a new log
I20260806 16:18:03.948428 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:03.930912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48380 (local address 127.11.245.190:34485)
0806 16:18:03.931485 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:03.931517 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:03.931572 (+ 55us) server_negotiation.cc:407] Connection header received
0806 16:18:03.932578 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:03.932608 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:03.933147 (+ 539us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:03.933602 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:18:03.934451 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:03.936027 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:03.937039 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:03.937830 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:03.938017 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:03.941529 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:18:03.941536 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:18:03.941557 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:18:03.941591 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:18:03.943999 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:18:03.944756 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:18:03.944763 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:18:03.944770 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:18:03.944931 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:18:03.945602 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:18:03.945605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:18:03.945607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:18:03.946205 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:18:03.946399 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:03.946806 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 16:18:03.947256 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":166,"threads_started":1}
I20260806 16:18:03.949641 27364 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c40277484f4575be4fe8e5899caad2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34485 } }
I20260806 16:18:03.950021 27364 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:18:03.950111 27364 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c40277484f4575be4fe8e5899caad2, State: Initialized, Role: FOLLOWER
I20260806 16:18:03.950686 27364 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c40277484f4575be4fe8e5899caad2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34485 } }
I20260806 16:18:03.950873 27364 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:18:03.950947 27364 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:18:03.951064 27364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:18:03.952608 27364 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c40277484f4575be4fe8e5899caad2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34485 } }
I20260806 16:18:03.953081 27364 leader_election.cc:302] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10c40277484f4575be4fe8e5899caad2; no voters:
I20260806 16:18:03.953557 27364 leader_election.cc:288] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:18:03.953725 27372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:18:03.954032 27372 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [term 1 LEADER]: Becoming Leader. State: Replica: 10c40277484f4575be4fe8e5899caad2, State: Running, Role: LEADER
I20260806 16:18:03.955241 27364 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:18:03.955240 27372 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c40277484f4575be4fe8e5899caad2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34485 } }
I20260806 16:18:03.958015 27374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10c40277484f4575be4fe8e5899caad2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c40277484f4575be4fe8e5899caad2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34485 } } }
I20260806 16:18:03.958123 27377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10c40277484f4575be4fe8e5899caad2. Latest consensus state: current_term: 1 leader_uuid: "10c40277484f4575be4fe8e5899caad2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c40277484f4575be4fe8e5899caad2" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34485 } } }
I20260806 16:18:03.958338 27374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [sys.catalog]: This master's current role is: LEADER
I20260806 16:18:03.958380 27377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2 [sys.catalog]: This master's current role is: LEADER
W20260806 16:18:03.963889 27395 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:18:03.964048 27395 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:18:03.964231 27397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:18:03.965422 27397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:18:03.973718 27397 catalog_manager.cc:1347] Generated new cluster ID: 8c66d920799349dd9bd09a2a6930e60a
I20260806 16:18:03.973873 27397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:18:03.987918 27397 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:18:03.989441 27397 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:18:04.002254 27397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2: Generated new TSK 0
I20260806 16:18:04.003777 27397 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:18:04.052366 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34485
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:18:04.202447 27404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:18:04.202869 27404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:18:04.202958 27404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:18:04.207787 27404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:18:04.207949 27404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:18:04.208091 27404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:18:04.214643 27404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:18:04.216365 27404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:18:04.217821 27404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:18:04.230780 27411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:04.230738 27410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:04.230790 27413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:04.231103 27404 server_base.cc:1034] running on GCE node
I20260806 16:18:04.231761 27404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:18:04.232745 27404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:18:04.233973 27404 hybrid_clock.cc:648] HybridClock initialized: now 1786033084233939 us; error 68 us; skew 500 ppm
Aug 06 16:18:04 dist-test-slave-t3q3 krb5kdc[27249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033084, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:18:04.237963 27404 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:18:04.239337 27404 webserver.cc:460] Webserver started at http://127.11.245.129:43395/ using document root <none> and password file <none>
I20260806 16:18:04.240183 27404 fs_manager.cc:359] Metadata directory not provided
I20260806 16:18:04.240290 27404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:18:04.240600 27404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:18:04.243361 27404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "9d5f00afdca14a32bf96389d0f51ad79"
format_stamp: "Formatted at 2026-08-06 16:18:04 on dist-test-slave-t3q3"
server_key: "b8b0814f7c8b418ac9b0ecce7962cfef"
server_key_iv: "dc484afe1baad360e41f4f9f13b93f40"
server_key_version: "encryptionkey@0"
I20260806 16:18:04.244181 27404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "9d5f00afdca14a32bf96389d0f51ad79"
format_stamp: "Formatted at 2026-08-06 16:18:04 on dist-test-slave-t3q3"
server_key: "b8b0814f7c8b418ac9b0ecce7962cfef"
server_key_iv: "dc484afe1baad360e41f4f9f13b93f40"
server_key_version: "encryptionkey@0"
I20260806 16:18:04.249451 27404 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 16:18:04.253089 27422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:04.254788 27404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 16:18:04.254976 27404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "9d5f00afdca14a32bf96389d0f51ad79"
format_stamp: "Formatted at 2026-08-06 16:18:04 on dist-test-slave-t3q3"
server_key: "b8b0814f7c8b418ac9b0ecce7962cfef"
server_key_iv: "dc484afe1baad360e41f4f9f13b93f40"
server_key_version: "encryptionkey@0"
I20260806 16:18:04.255147 27404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:18:04.270913 27404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:18:04.274844 27404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:18:04.275106 27404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:18:04.275936 27404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:18:04.277100 27404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:18:04.277182 27404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:04.277271 27404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:18:04.277319 27404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:04.289033 27404 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:45075
I20260806 16:18:04.289067 27589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:45075 every 8 connection(s)
I20260806 16:18:04.290284 27404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:18:04.294520 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 27404
I20260806 16:18:04.294672 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:18:04.295042 12246 external_mini_cluster.cc:1428] Setting key 929aab6556a16ba0e39ac6e45348e5c5
Aug 06 16:18:04 dist-test-slave-t3q3 krb5kdc[27249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033084, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:18:04.309437 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:04.292813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:44951 (local address 127.11.245.190:34485)
0806 16:18:04.293018 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:04.293032 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:04.294134 (+ 1102us) server_negotiation.cc:407] Connection header received
0806 16:18:04.295972 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:04.295975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:04.296030 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:04.296181 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:18:04.297828 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.298425 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.300029 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.300336 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.300426 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:04.304029 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:18:04.304040 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:18:04.304044 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:18:04.304103 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:18:04.305995 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:18:04.306826 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:18:04.306830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:18:04.306831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:18:04.306931 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:18:04.307435 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:18:04.307438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:18:04.307439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:18:04.307998 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:18:04.308206 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:04.309081 (+ 875us) server_negotiation.cc:299] Negotiation successful
0806 16:18:04.309250 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 16:18:04.310693 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:04.293328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:34485 (local address 127.11.245.129:44951)
0806 16:18:04.293936 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 16:18:04.293991 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 16:18:04.295724 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:18:04.296447 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:18:04.296457 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:18:04.296961 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:18:04.297635 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:04.297653 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.298585 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.298588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:04.299869 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:04.299885 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.300517 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.300523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:04.300774 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:04.301573 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:18:04.301601 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:18:04.303768 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:18:04.306172 (+ 2404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:18:04.306183 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:18:04.306210 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:18:04.306665 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:18:04.307058 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:18:04.307062 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:18:04.307065 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:18:04.307307 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:18:04.308188 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:18:04.308201 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:18:04.308702 (+ 501us) client_negotiation.cc:769] Sending connection context
0806 16:18:04.309015 (+ 313us) client_negotiation.cc:240] Negotiation successful
0806 16:18:04.309468 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":130,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
I20260806 16:18:04.312588 27592 heartbeater.cc:347] Connected to a master server at 127.11.245.190:34485
I20260806 16:18:04.312925 27592 heartbeater.cc:464] Registering TS with master...
I20260806 16:18:04.313663 27592 heartbeater.cc:511] Master 127.11.245.190:34485 requested a full tablet report, sending...
I20260806 16:18:04.316258 27313 ts_manager.cc:194] Registered new tserver with Master: 9d5f00afdca14a32bf96389d0f51ad79 (127.11.245.129:45075)
I20260806 16:18:04.317817 27313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:44951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:18:04.332151 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34485
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:18:04.479496 27600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:18:04.479892 27600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:18:04.479992 27600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:18:04.484602 27600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:18:04.484699 27600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:18:04.484800 27600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:18:04.491621 27600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:18:04.493330 27600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:18:04.494832 27600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:18:04.509229 27607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:04.509213 27611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:04.509210 27608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:04.509668 27600 server_base.cc:1034] running on GCE node
I20260806 16:18:04.510434 27600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:18:04.511519 27600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:18:04.512764 27600 hybrid_clock.cc:648] HybridClock initialized: now 1786033084512740 us; error 42 us; skew 500 ppm
Aug 06 16:18:04 dist-test-slave-t3q3 krb5kdc[27249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033084, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:18:04.516686 27600 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:18:04.517952 27600 webserver.cc:460] Webserver started at http://127.11.245.130:40587/ using document root <none> and password file <none>
I20260806 16:18:04.518730 27600 fs_manager.cc:359] Metadata directory not provided
I20260806 16:18:04.518832 27600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:18:04.519142 27600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:18:04.521975 27600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "bb0b63ee250b4c38a4ad949541138197"
format_stamp: "Formatted at 2026-08-06 16:18:04 on dist-test-slave-t3q3"
server_key: "e7251d49ee7c644c54a3e872b1b5b0c7"
server_key_iv: "ca648a784f7b2366e82bfb440fb82005"
server_key_version: "encryptionkey@0"
I20260806 16:18:04.522832 27600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "bb0b63ee250b4c38a4ad949541138197"
format_stamp: "Formatted at 2026-08-06 16:18:04 on dist-test-slave-t3q3"
server_key: "e7251d49ee7c644c54a3e872b1b5b0c7"
server_key_iv: "ca648a784f7b2366e82bfb440fb82005"
server_key_version: "encryptionkey@0"
I20260806 16:18:04.528654 27600 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:18:04.532433 27620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:04.534152 27600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:18:04.534348 27600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "bb0b63ee250b4c38a4ad949541138197"
format_stamp: "Formatted at 2026-08-06 16:18:04 on dist-test-slave-t3q3"
server_key: "e7251d49ee7c644c54a3e872b1b5b0c7"
server_key_iv: "ca648a784f7b2366e82bfb440fb82005"
server_key_version: "encryptionkey@0"
I20260806 16:18:04.534543 27600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:18:04.567564 27600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:18:04.571976 27600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:18:04.572280 27600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:18:04.573204 27600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:18:04.574416 27600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:18:04.574499 27600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:04.574586 27600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:18:04.574643 27600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:04.585932 27600 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:38739
I20260806 16:18:04.585961 27786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:38739 every 8 connection(s)
I20260806 16:18:04.587251 27600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:18:04.593895 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 27600
I20260806 16:18:04.594035 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:18:04.594430 12246 external_mini_cluster.cc:1428] Setting key cd0f3763c4564e667e89c2589b9f9aed
Aug 06 16:18:04 dist-test-slave-t3q3 krb5kdc[27249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033084, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:18:04.606788 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:04.590010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:47339 (local address 127.11.245.190:34485)
0806 16:18:04.590242 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:04.590247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:04.591714 (+ 1467us) server_negotiation.cc:407] Connection header received
0806 16:18:04.593574 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:04.593578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:04.593658 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:04.593783 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:18:04.595635 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.596384 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.597783 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.598038 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.598162 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:04.601999 (+ 3837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:18:04.602006 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:18:04.602012 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:18:04.602052 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:18:04.604096 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:18:04.604710 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:18:04.604719 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:18:04.604722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:18:04.604834 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:18:04.605225 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:18:04.605231 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:18:04.605236 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:18:04.605574 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:18:04.605702 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:04.606378 (+ 676us) server_negotiation.cc:299] Negotiation successful
0806 16:18:04.606550 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260806 16:18:04.607959 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:04.590771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:34485 (local address 127.11.245.130:47339)
0806 16:18:04.591481 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0806 16:18:04.591540 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 16:18:04.593267 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:18:04.594066 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:18:04.594084 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:18:04.594653 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:18:04.595449 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:04.595486 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.596543 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.596547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:04.597634 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:04.597646 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.598235 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.598239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:04.598455 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:04.599400 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:18:04.599433 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:18:04.601769 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:18:04.604225 (+ 2456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:18:04.604232 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:18:04.604251 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:18:04.604587 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:18:04.604942 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:18:04.604944 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:18:04.604946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:18:04.605118 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:18:04.605699 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:18:04.605706 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:18:04.606060 (+ 354us) client_negotiation.cc:769] Sending connection context
0806 16:18:04.606273 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 16:18:04.606654 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":172,"threads_started":1}
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
I20260806 16:18:04.609673 27788 heartbeater.cc:347] Connected to a master server at 127.11.245.190:34485
I20260806 16:18:04.610047 27788 heartbeater.cc:464] Registering TS with master...
I20260806 16:18:04.610746 27788 heartbeater.cc:511] Master 127.11.245.190:34485 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
I20260806 16:18:04.612432 27313 ts_manager.cc:194] Registered new tserver with Master: bb0b63ee250b4c38a4ad949541138197 (127.11.245.130:38739)
I20260806 16:18:04.613240 27313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:47339
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:18:04.632923 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34485
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:18:04.782752 27794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:18:04.783134 27794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:18:04.783190 27794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:18:04.787665 27794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:18:04.787770 27794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:18:04.787840 27794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:18:04.794340 27794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:18:04.795950 27794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:18:04.797322 27794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:18:04.810274 27805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:04.810564 27794 server_base.cc:1034] running on GCE node
W20260806 16:18:04.810652 27801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:04.810274 27799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:04.811260 27794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:18:04.812659 27794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:18:04.813901 27794 hybrid_clock.cc:648] HybridClock initialized: now 1786033084813867 us; error 56 us; skew 500 ppm
Aug 06 16:18:04 dist-test-slave-t3q3 krb5kdc[27249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033084, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:18:04.817772 27794 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:18:04.819571 27794 webserver.cc:460] Webserver started at http://127.11.245.131:32857/ using document root <none> and password file <none>
I20260806 16:18:04.820511 27794 fs_manager.cc:359] Metadata directory not provided
I20260806 16:18:04.820634 27794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:18:04.820953 27794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:18:04.824385 27794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "5dee97f5e556497b9afecb9d3096eadf"
format_stamp: "Formatted at 2026-08-06 16:18:04 on dist-test-slave-t3q3"
server_key: "e15ec77011544e767ba6664c215d2d55"
server_key_iv: "f3c6a29cc9d3dc5266a1c0691c405fa1"
server_key_version: "encryptionkey@0"
I20260806 16:18:04.825176 27794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "5dee97f5e556497b9afecb9d3096eadf"
format_stamp: "Formatted at 2026-08-06 16:18:04 on dist-test-slave-t3q3"
server_key: "e15ec77011544e767ba6664c215d2d55"
server_key_iv: "f3c6a29cc9d3dc5266a1c0691c405fa1"
server_key_version: "encryptionkey@0"
I20260806 16:18:04.830683 27794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 16:18:04.834359 27814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:04.836303 27794 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:18:04.836488 27794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "5dee97f5e556497b9afecb9d3096eadf"
format_stamp: "Formatted at 2026-08-06 16:18:04 on dist-test-slave-t3q3"
server_key: "e15ec77011544e767ba6664c215d2d55"
server_key_iv: "f3c6a29cc9d3dc5266a1c0691c405fa1"
server_key_version: "encryptionkey@0"
I20260806 16:18:04.836658 27794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:18:04.859457 27794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:18:04.863955 27794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:18:04.864293 27794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:18:04.865209 27794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:18:04.866415 27794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:18:04.866500 27794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:04.866619 27794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:18:04.866668 27794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:04.878907 27794 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:44695
I20260806 16:18:04.878933 27981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:44695 every 8 connection(s)
I20260806 16:18:04.880292 27794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:18:04.884508 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 27794
I20260806 16:18:04.884689 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:18:04.885124 12246 external_mini_cluster.cc:1428] Setting key cb74ed5a3b7e645c518c4c660b77077f
Aug 06 16:18:04 dist-test-slave-t3q3 krb5kdc[27249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033084, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:18:04.898994 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:04.882894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:51779 (local address 127.11.245.190:34485)
0806 16:18:04.883091 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:04.883096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:04.884440 (+ 1344us) server_negotiation.cc:407] Connection header received
0806 16:18:04.886466 (+ 2026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:04.886469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:04.886526 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:04.886640 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:18:04.888595 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.889187 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.890625 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.890923 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.891012 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:04.894259 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:18:04.894263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:18:04.894265 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:18:04.894295 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:18:04.895852 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:18:04.896625 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:18:04.896629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:18:04.896630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:18:04.896728 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:18:04.897168 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:18:04.897171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:18:04.897172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:18:04.897489 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:18:04.897599 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:04.898582 (+ 983us) server_negotiation.cc:299] Negotiation successful
0806 16:18:04.898792 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 16:18:04.900288 27987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:04.883465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:34485 (local address 127.11.245.131:51779)
0806 16:18:04.884172 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0806 16:18:04.884247 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 16:18:04.886204 (+ 1957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:18:04.886990 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:18:04.887000 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:18:04.887580 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:18:04.888441 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:04.888461 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.889345 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.889348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:04.890472 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:04.890484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.891106 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.891109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:04.891366 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:04.892201 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:18:04.892221 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:18:04.894063 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:18:04.896006 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:18:04.896013 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:18:04.896033 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:18:04.896471 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:18:04.896833 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:18:04.896835 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:18:04.896837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:18:04.897024 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:18:04.897669 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:18:04.897677 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:18:04.898132 (+ 455us) client_negotiation.cc:769] Sending connection context
0806 16:18:04.898436 (+ 304us) client_negotiation.cc:240] Negotiation successful
0806 16:18:04.898917 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":147,"threads_started":1}
I20260806 16:18:04.901842 27984 heartbeater.cc:347] Connected to a master server at 127.11.245.190:34485
I20260806 16:18:04.902349 27984 heartbeater.cc:464] Registering TS with master...
I20260806 16:18:04.903144 27984 heartbeater.cc:511] Master 127.11.245.190:34485 requested a full tablet report, sending...
I20260806 16:18:04.904793 27313 ts_manager.cc:194] Registered new tserver with Master: 5dee97f5e556497b9afecb9d3096eadf (127.11.245.131:44695)
I20260806 16:18:04.905696 27313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:51779
I20260806 16:18:04.912792 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:18:04.924392 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:04.915448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48396 (local address 127.11.245.190:34485)
0806 16:18:04.915639 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:04.915645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:04.915862 (+ 217us) server_negotiation.cc:407] Connection header received
0806 16:18:04.915998 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:04.916001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:04.916094 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:04.916198 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:18:04.917019 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.917624 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.918555 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.918797 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.918920 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:04.920616 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:18:04.920620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:18:04.920622 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:18:04.920649 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:18:04.922441 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:18:04.922943 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:18:04.922947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:18:04.922949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:18:04.923067 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:18:04.923358 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:18:04.923362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:18:04.923363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:18:04.923677 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:18:04.923829 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:04.924053 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 16:18:04.924218 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 16:18:04.928295 27313 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:18:04.933549 27313 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:18:04.953924 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:04.946491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:44695 (local address 127.0.0.1:40390)
0806 16:18:04.947377 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0806 16:18:04.947398 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:18:04.947519 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:18:04.948538 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:18:04.948541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:18:04.948555 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:18:04.948650 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:04.948654 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.950158 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.950162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:04.951925 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:04.951937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.953410 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.953414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:04.953579 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:04.953610 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 16:18:04.953682 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 16:18:04.953761 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"mutex_wait_us":209,"thread_start_us":161,"threads_started":1}
I20260806 16:18:04.953969 28005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:04.946617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59476 (local address 127.11.245.129:45075)
0806 16:18:04.947609 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:04.947619 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:04.947654 (+ 35us) server_negotiation.cc:407] Connection header received
0806 16:18:04.947760 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:04.947766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:04.948044 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:04.948740 (+ 696us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:18:04.948911 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.950128 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.952101 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.952934 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.953112 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:04.953251 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:04.953635 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 16:18:04.953766 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":845,"thread_start_us":105,"threads_started":1}
I20260806 16:18:04.953928 28004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:04.946420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:45075 (local address 127.0.0.1:59476)
0806 16:18:04.947065 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0806 16:18:04.947127 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 16:18:04.947279 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:18:04.948309 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:18:04.948317 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:18:04.948344 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:18:04.948561 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:04.948570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.950285 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.950288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:04.951925 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:04.951937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.953093 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.953097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:04.953225 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:04.953276 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 16:18:04.953422 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 16:18:04.953699 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":134,"thread_start_us":147,"threads_started":1}
I20260806 16:18:04.954627 28006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:04.947066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40390 (local address 127.11.245.131:44695)
0806 16:18:04.947671 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:04.947686 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:04.947712 (+ 26us) server_negotiation.cc:407] Connection header received
0806 16:18:04.947816 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:04.947824 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:04.948053 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:04.948303 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:18:04.948780 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.949994 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.952116 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.953069 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.953200 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:04.953336 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:04.954329 (+ 993us) server_negotiation.cc:299] Negotiation successful
0806 16:18:04.954454 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":130,"threads_started":1}
I20260806 16:18:04.955760 28007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:04.947706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:38739 (local address 127.0.0.1:57404)
0806 16:18:04.948258 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0806 16:18:04.948272 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:18:04.948364 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:18:04.949129 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:18:04.949133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:18:04.949153 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:18:04.949264 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:04.949271 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.950710 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.950716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:04.951563 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:04.951573 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.953315 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.953318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:04.955421 (+ 2103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:04.955450 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 16:18:04.955542 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 16:18:04.955629 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":1942,"thread_start_us":115,"threads_started":1}
I20260806 16:18:04.956034 28008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:04.947954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57404 (local address 127.11.245.130:38739)
0806 16:18:04.948610 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:04.948627 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:04.948655 (+ 28us) server_negotiation.cc:407] Connection header received
0806 16:18:04.948746 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:04.948751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:04.948953 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:04.949128 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:18:04.949517 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.950566 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.951705 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:04.952575 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:04.952718 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:04.952898 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:04.955616 (+ 2718us) server_negotiation.cc:299] Negotiation successful
0806 16:18:04.955741 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":267,"threads_started":1}
I20260806 16:18:04.958201 27882 tablet_service.cc:1467] Processing CreateTablet for tablet fb9965dbea90438f933cca35ecfd8041 (DEFAULT_TABLE table=test-table [id=bffea2c939d64fa1aa961e5e1a442c14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:18:04.958205 27491 tablet_service.cc:1467] Processing CreateTablet for tablet fb9965dbea90438f933cca35ecfd8041 (DEFAULT_TABLE table=test-table [id=bffea2c939d64fa1aa961e5e1a442c14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:18:04.959559 27882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb9965dbea90438f933cca35ecfd8041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:18:04.959558 27491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb9965dbea90438f933cca35ecfd8041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:18:04.959297 27684 tablet_service.cc:1467] Processing CreateTablet for tablet fb9965dbea90438f933cca35ecfd8041 (DEFAULT_TABLE table=test-table [id=bffea2c939d64fa1aa961e5e1a442c14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:18:04.960465 27684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb9965dbea90438f933cca35ecfd8041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:18:04.969405 28013 tablet_bootstrap.cc:492] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf: Bootstrap starting.
I20260806 16:18:04.972165 28013 tablet_bootstrap.cc:654] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf: Neither blocks nor log segments found. Creating new log.
I20260806 16:18:04.972597 28010 tablet_bootstrap.cc:492] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79: Bootstrap starting.
I20260806 16:18:04.972801 28012 tablet_bootstrap.cc:492] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197: Bootstrap starting.
I20260806 16:18:04.973744 28013 log.cc:824] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf: Log is configured to *not* fsync() on all Append() calls
I20260806 16:18:04.975664 28010 tablet_bootstrap.cc:654] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79: Neither blocks nor log segments found. Creating new log.
I20260806 16:18:04.976255 28012 tablet_bootstrap.cc:654] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197: Neither blocks nor log segments found. Creating new log.
I20260806 16:18:04.976889 28010 log.cc:824] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79: Log is configured to *not* fsync() on all Append() calls
I20260806 16:18:04.977185 28012 log.cc:824] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197: Log is configured to *not* fsync() on all Append() calls
I20260806 16:18:04.977432 28013 tablet_bootstrap.cc:492] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf: No bootstrap required, opened a new log
I20260806 16:18:04.977717 28013 ts_tablet_manager.cc:1397] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260806 16:18:04.979490 28010 tablet_bootstrap.cc:492] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79: No bootstrap required, opened a new log
I20260806 16:18:04.979627 28012 tablet_bootstrap.cc:492] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197: No bootstrap required, opened a new log
I20260806 16:18:04.979790 28010 ts_tablet_manager.cc:1397] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260806 16:18:04.979853 28012 ts_tablet_manager.cc:1397] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 16:18:04.982990 28013 raft_consensus.cc:348] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5f00afdca14a32bf96389d0f51ad79" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45075 } } peers { permanent_uuid: "5dee97f5e556497b9afecb9d3096eadf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44695 } } peers { permanent_uuid: "bb0b63ee250b4c38a4ad949541138197" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 38739 } }
I20260806 16:18:04.983425 28013 raft_consensus.cc:374] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:18:04.983521 28013 raft_consensus.cc:729] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dee97f5e556497b9afecb9d3096eadf, State: Initialized, Role: FOLLOWER
I20260806 16:18:04.984236 28013 consensus_queue.cc:260] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5f00afdca14a32bf96389d0f51ad79" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45075 } } peers { permanent_uuid: "5dee97f5e556497b9afecb9d3096eadf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44695 } } peers { permanent_uuid: "bb0b63ee250b4c38a4ad949541138197" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 38739 } }
I20260806 16:18:04.984275 28010 raft_consensus.cc:348] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5f00afdca14a32bf96389d0f51ad79" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45075 } } peers { permanent_uuid: "5dee97f5e556497b9afecb9d3096eadf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44695 } } peers { permanent_uuid: "bb0b63ee250b4c38a4ad949541138197" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 38739 } }
I20260806 16:18:04.984594 28010 raft_consensus.cc:374] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:18:04.984668 28010 raft_consensus.cc:729] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d5f00afdca14a32bf96389d0f51ad79, State: Initialized, Role: FOLLOWER
I20260806 16:18:04.984927 28012 raft_consensus.cc:348] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5f00afdca14a32bf96389d0f51ad79" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45075 } } peers { permanent_uuid: "5dee97f5e556497b9afecb9d3096eadf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44695 } } peers { permanent_uuid: "bb0b63ee250b4c38a4ad949541138197" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 38739 } }
I20260806 16:18:04.985226 28012 raft_consensus.cc:374] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:18:04.985291 28012 raft_consensus.cc:729] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb0b63ee250b4c38a4ad949541138197, State: Initialized, Role: FOLLOWER
I20260806 16:18:04.985255 28010 consensus_queue.cc:260] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5f00afdca14a32bf96389d0f51ad79" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45075 } } peers { permanent_uuid: "5dee97f5e556497b9afecb9d3096eadf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44695 } } peers { permanent_uuid: "bb0b63ee250b4c38a4ad949541138197" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 38739 } }
I20260806 16:18:04.985769 28012 consensus_queue.cc:260] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5f00afdca14a32bf96389d0f51ad79" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45075 } } peers { permanent_uuid: "5dee97f5e556497b9afecb9d3096eadf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44695 } } peers { permanent_uuid: "bb0b63ee250b4c38a4ad949541138197" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 38739 } }
I20260806 16:18:04.986409 28010 ts_tablet_manager.cc:1428] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 16:18:04.986410 28013 ts_tablet_manager.cc:1428] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260806 16:18:04.986760 27788 heartbeater.cc:503] Master 127.11.245.190:34485 was elected leader, sending a full tablet report...
I20260806 16:18:04.986788 27984 heartbeater.cc:503] Master 127.11.245.190:34485 was elected leader, sending a full tablet report...
I20260806 16:18:04.986826 27592 heartbeater.cc:503] Master 127.11.245.190:34485 was elected leader, sending a full tablet report...
I20260806 16:18:04.986774 28012 ts_tablet_manager.cc:1428] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260806 16:18:05.042148 27594 tablet.cc:2369] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:18:05.088992 27789 tablet.cc:2369] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:18:05.116907 28024 raft_consensus.cc:482] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:18:05.117219 28024 raft_consensus.cc:504] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5f00afdca14a32bf96389d0f51ad79" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45075 } } peers { permanent_uuid: "5dee97f5e556497b9afecb9d3096eadf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44695 } } peers { permanent_uuid: "bb0b63ee250b4c38a4ad949541138197" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 38739 } }
I20260806 16:18:05.118795 28024 leader_election.cc:288] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dee97f5e556497b9afecb9d3096eadf (127.11.245.131:44695), bb0b63ee250b4c38a4ad949541138197 (127.11.245.130:38739)
I20260806 16:18:05.123472 28026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:05.119248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:44695 (local address 127.11.245.129:57073)
0806 16:18:05.119663 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0806 16:18:05.119681 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:18:05.119853 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:18:05.120265 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:18:05.120269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:18:05.120294 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:18:05.120461 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:05.120468 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:05.121287 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:05.121290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:05.122207 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:05.122217 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:05.122945 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:05.122949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:05.123064 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:05.123098 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 16:18:05.123191 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 16:18:05.123276 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":187,"threads_started":1}
I20260806 16:18:05.123456 28006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:05.119346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:57073 (local address 127.11.245.131:44695)
0806 16:18:05.119536 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:05.119540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:05.119815 (+ 275us) server_negotiation.cc:407] Connection header received
0806 16:18:05.120018 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:05.120022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:05.120121 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:05.120220 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:18:05.120595 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:05.121162 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:05.122347 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:05.122795 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:05.122959 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:05.123036 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:05.123212 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 16:18:05.123301 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 16:18:05.124050 28008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:05.120116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:57257 (local address 127.11.245.130:38739)
0806 16:18:05.120310 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:05.120315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:05.120433 (+ 118us) server_negotiation.cc:407] Connection header received
0806 16:18:05.120604 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:05.120614 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:05.120669 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:05.120760 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:18:05.121056 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:05.121683 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:05.122787 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:05.123216 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:05.123356 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:05.123416 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:05.123817 (+ 401us) server_negotiation.cc:299] Negotiation successful
0806 16:18:05.123925 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 16:18:05.124032 28027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:05.119846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:38739 (local address 127.11.245.129:57257)
0806 16:18:05.120324 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 16:18:05.120339 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:18:05.120481 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:18:05.120775 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:18:05.120779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:18:05.120797 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:18:05.120936 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:05.120942 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:05.121856 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:05.121859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:05.122652 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:18:05.122662 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:18:05.123346 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:18:05.123350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:18:05.123586 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:05.123625 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 16:18:05.123794 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 16:18:05.123878 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":126,"threads_started":1}
I20260806 16:18:05.124542 27917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9965dbea90438f933cca35ecfd8041" candidate_uuid: "9d5f00afdca14a32bf96389d0f51ad79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dee97f5e556497b9afecb9d3096eadf" is_pre_election: true
I20260806 16:18:05.124864 27715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9965dbea90438f933cca35ecfd8041" candidate_uuid: "9d5f00afdca14a32bf96389d0f51ad79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb0b63ee250b4c38a4ad949541138197" is_pre_election: true
I20260806 16:18:05.125034 27917 raft_consensus.cc:2454] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d5f00afdca14a32bf96389d0f51ad79 in term 0.
I20260806 16:18:05.125218 27715 raft_consensus.cc:2454] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d5f00afdca14a32bf96389d0f51ad79 in term 0.
I20260806 16:18:05.125626 27424 leader_election.cc:302] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dee97f5e556497b9afecb9d3096eadf, 9d5f00afdca14a32bf96389d0f51ad79; no voters:
I20260806 16:18:05.125993 28024 raft_consensus.cc:2781] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:18:05.126106 28024 raft_consensus.cc:482] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:18:05.126190 28024 raft_consensus.cc:3037] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:18:05.127588 28024 raft_consensus.cc:504] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5f00afdca14a32bf96389d0f51ad79" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45075 } } peers { permanent_uuid: "5dee97f5e556497b9afecb9d3096eadf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44695 } } peers { permanent_uuid: "bb0b63ee250b4c38a4ad949541138197" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 38739 } }
I20260806 16:18:05.128010 28024 leader_election.cc:288] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [CANDIDATE]: Term 1 election: Requested vote from peers 5dee97f5e556497b9afecb9d3096eadf (127.11.245.131:44695), bb0b63ee250b4c38a4ad949541138197 (127.11.245.130:38739)
I20260806 16:18:05.128618 27917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9965dbea90438f933cca35ecfd8041" candidate_uuid: "9d5f00afdca14a32bf96389d0f51ad79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dee97f5e556497b9afecb9d3096eadf"
I20260806 16:18:05.128618 27715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9965dbea90438f933cca35ecfd8041" candidate_uuid: "9d5f00afdca14a32bf96389d0f51ad79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb0b63ee250b4c38a4ad949541138197"
I20260806 16:18:05.128778 27715 raft_consensus.cc:3037] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:18:05.128779 27917 raft_consensus.cc:3037] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:18:05.130393 27715 raft_consensus.cc:2454] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d5f00afdca14a32bf96389d0f51ad79 in term 1.
I20260806 16:18:05.130393 27917 raft_consensus.cc:2454] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d5f00afdca14a32bf96389d0f51ad79 in term 1.
I20260806 16:18:05.130723 27424 leader_election.cc:302] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d5f00afdca14a32bf96389d0f51ad79, bb0b63ee250b4c38a4ad949541138197; no voters:
I20260806 16:18:05.131163 28024 raft_consensus.cc:2781] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [term 1 FOLLOWER]: Leader election won for term 1
W20260806 16:18:05.131669 27986 tablet.cc:2369] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:18:05.131645 28024 raft_consensus.cc:686] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [term 1 LEADER]: Becoming Leader. State: Replica: 9d5f00afdca14a32bf96389d0f51ad79, State: Running, Role: LEADER
I20260806 16:18:05.132269 28024 consensus_queue.cc:237] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5f00afdca14a32bf96389d0f51ad79" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45075 } } peers { permanent_uuid: "5dee97f5e556497b9afecb9d3096eadf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44695 } } peers { permanent_uuid: "bb0b63ee250b4c38a4ad949541138197" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 38739 } }
I20260806 16:18:05.136518 27313 catalog_manager.cc:5495] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d5f00afdca14a32bf96389d0f51ad79 (127.11.245.129). New cstate: current_term: 1 leader_uuid: "9d5f00afdca14a32bf96389d0f51ad79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d5f00afdca14a32bf96389d0f51ad79" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dee97f5e556497b9afecb9d3096eadf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb0b63ee250b4c38a4ad949541138197" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 38739 } health_report { overall_health: UNKNOWN } } }
I20260806 16:18:05.197077 28005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:05.192584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59478 (local address 127.11.245.129:45075)
0806 16:18:05.192767 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:05.192771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:05.192785 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:18:05.192894 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:05.192897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:05.192989 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:05.193111 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:18:05.193474 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:05.194087 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:05.195105 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:05.195367 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:05.195466 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:05.195896 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:18:05.196674 (+ 778us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:18:05.196812 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:05.196844 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 16:18:05.196918 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 16:18:05.205560 27917 raft_consensus.cc:1261] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf [term 1 FOLLOWER]: Refusing update from remote peer 9d5f00afdca14a32bf96389d0f51ad79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:18:05.205560 27715 raft_consensus.cc:1261] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197 [term 1 FOLLOWER]: Refusing update from remote peer 9d5f00afdca14a32bf96389d0f51ad79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:18:05.206534 28024 consensus_queue.cc:1035] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb0b63ee250b4c38a4ad949541138197" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 38739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:18:05.206809 28029 consensus_queue.cc:1035] T fb9965dbea90438f933cca35ecfd8041 P 9d5f00afdca14a32bf96389d0f51ad79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dee97f5e556497b9afecb9d3096eadf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 44695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:18:05.215806 28036 mvcc.cc:205] Tried to move back new op lower bound from 7315591516991234048 to 7315591516705427456. Current Snapshot: MvccSnapshot[applied={T|T < 7315591516991234048}]
I20260806 16:18:05.227331 27310 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:18:05.227574 27310 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:18:05.231140 27310 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10c40277484f4575be4fe8e5899caad2: Sending DeleteTablet for 3 replicas of tablet fb9965dbea90438f933cca35ecfd8041
I20260806 16:18:05.232646 27491 tablet_service.cc:1514] Processing DeleteTablet for tablet fb9965dbea90438f933cca35ecfd8041 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:18:05 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:59476
I20260806 16:18:05.233002 27882 tablet_service.cc:1514] Processing DeleteTablet for tablet fb9965dbea90438f933cca35ecfd8041 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:18:05 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:40390
I20260806 16:18:05.233001 27684 tablet_service.cc:1514] Processing DeleteTablet for tablet fb9965dbea90438f933cca35ecfd8041 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:18:05 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:57404
I20260806 16:18:05.233707 28048 tablet_replica.cc:330] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf: stopping tablet replica
I20260806 16:18:05.234673 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 27404
I20260806 16:18:05.238263 28048 raft_consensus.cc:2229] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:18:05.239053 28048 raft_consensus.cc:2258] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:18:05.246994 28049 tablet_replica.cc:330] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197: stopping tablet replica
I20260806 16:18:05.247578 28049 raft_consensus.cc:2229] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:18:05.247754 28048 ts_tablet_manager.cc:1905] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:18:05.248169 28049 raft_consensus.cc:2258] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:18:05.250382 28049 ts_tablet_manager.cc:1905] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:18:05.250991 28048 ts_tablet_manager.cc:1918] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:18:05.251132 28048 log.cc:1196] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/fb9965dbea90438f933cca35ecfd8041
I20260806 16:18:05.251507 28048 ts_tablet_manager.cc:1939] T fb9965dbea90438f933cca35ecfd8041 P 5dee97f5e556497b9afecb9d3096eadf: Deleting consensus metadata
I20260806 16:18:05.253217 27291 catalog_manager.cc:4841] TS 5dee97f5e556497b9afecb9d3096eadf (127.11.245.131:44695): tablet fb9965dbea90438f933cca35ecfd8041 (table test-table [id=bffea2c939d64fa1aa961e5e1a442c14]) successfully deleted
I20260806 16:18:05.254455 28049 ts_tablet_manager.cc:1918] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:18:05.254585 28049 log.cc:1196] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/fb9965dbea90438f933cca35ecfd8041
I20260806 16:18:05.254979 28049 ts_tablet_manager.cc:1939] T fb9965dbea90438f933cca35ecfd8041 P bb0b63ee250b4c38a4ad949541138197: Deleting consensus metadata
I20260806 16:18:05.256754 27291 catalog_manager.cc:4841] TS bb0b63ee250b4c38a4ad949541138197 (127.11.245.130:38739): tablet fb9965dbea90438f933cca35ecfd8041 (table test-table [id=bffea2c939d64fa1aa961e5e1a442c14]) successfully deleted
W20260806 16:18:05.258095 27293 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:45075 (error 108)
I20260806 16:18:05.259298 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 27600
I20260806 16:18:05.259564 28004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:05.259103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:45075 (local address 127.0.0.1:59482)
0806 16:18:05.259313 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 16:18:05.259420 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:45075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260806 16:18:05.262957 27293 catalog_manager.cc:4584] TS 9d5f00afdca14a32bf96389d0f51ad79 (127.11.245.129:45075): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fb9965dbea90438f933cca35ecfd8041: Network error: Client connection negotiation failed: client connection to 127.11.245.129:45075: connect: Connection refused (error 111)
I20260806 16:18:05.278492 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 27794
I20260806 16:18:05.292886 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 27270
2026-08-06T16:18:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (3081 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:18:05 dist-test-slave-t3q3 krb5kdc[28054](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:18:05 dist-test-slave-t3q3 krb5kdc[28054](info): set up 2 sockets
Aug 06 16:18:05 dist-test-slave-t3q3 krb5kdc[28054](info): commencing operation
krb5kdc: starting...
W20260806 16:18:06.618782 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.001s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:18:06 dist-test-slave-t3q3 krb5kdc[28054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:18:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:18:06Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.245.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.245.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.245.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/rangeradmin_127.11.245.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.245.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/rangeradmin_127.11.245.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.245.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.245.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.245.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/rangerlookup_127.11.245.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.245.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/rangerlookup_127.11.245.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.245.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.245.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.245.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/HTTP_127.11.245.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.245.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/HTTP_127.11.245.148@KRBTEST.COM.keytab.
I20260806 16:18:06.851192 12246 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskmdsaCi/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/postgres -l logfile start
2026-08-06 16:18:09.766 UTC [28101] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 16:18:09.766 UTC [28101] LOG: listening on IPv4 address "127.11.245.148", port 60545
2026-08-06 16:18:09.770 UTC [28101] LOG: listening on Unix socket "/tmp/.s.PGSQL.60545"
2026-08-06 16:18:09.794 UTC [28105] LOG: database system was shut down at 2026-08-06 16:18:08 UTC
2026-08-06 16:18:09.799 UTC [28101] LOG: database system is ready to accept connections
I20260806 16:18:10.991513 12246 mini_postgres.cc:96] Postgres bound to 60545
2026-08-06 16:18:11.001 UTC [28114] FATAL: database "slave" does not exist
127.11.245.148:60545 - accepting connections
I20260806 16:18:11.002640 12246 mini_ranger.cc:161] Starting Ranger...
I20260806 16:18:11.023488 12246 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 16:18:11.607996 12246 mini_ranger.cc:87] Created ranger Postgres database
I20260806 16:18:11.608192 12246 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-admin
2026-08-06 16:18:11,945 [I] DB FLAVOR :POSTGRES
2026-08-06 16:18:11,945 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 16:18:11,945 [I] Checking connection..
2026-08-06 16:18:11,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 16:18:12,256 [I] Checking connection passed.
2026-08-06 16:18:12,256 [I] --------- Verifying version history table ---------
2026-08-06 16:18:12,256 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:18:12,560 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 16:18:12,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:18:12,867 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 16:18:12,867 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 16:18:12,868 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 16:18:13.155 UTC [28206] WARNING: there is no transaction in progress
2026-08-06 16:18:13,165 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 16:18:13,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:18:13,471 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 16:18:13,471 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 16:18:13,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 16:18:13,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 16:18:14,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-t3q3.c.gcp-upstream.internal','N') ;"
2026-08-06 16:18:14,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 16:18:14,621 [I] Table x_portal_user does not exist in database ranger
2026-08-06 16:18:14,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 16:18:14,904 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 16:18:14,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 16:18:15,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 16:18:15,505 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 16:18:15,506 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 16:18:16.649 UTC [28365] WARNING: there is no transaction in progress
2026-08-06 16:18:16.668 UTC [28365] WARNING: there is no transaction in progress
2026-08-06 16:18:16.688 UTC [28365] WARNING: there is no transaction in progress
2026-08-06 16:18:16.707 UTC [28365] WARNING: there is no transaction in progress
2026-08-06 16:18:16.726 UTC [28365] WARNING: there is no transaction in progress
2026-08-06 16:18:16.749 UTC [28365] WARNING: there is no transaction in progress
2026-08-06 16:18:16.822 UTC [28365] WARNING: there is no transaction in progress
2026-08-06 16:18:16.830 UTC [28365] WARNING: there is no transaction in progress
2026-08-06 16:18:16.848 UTC [28365] WARNING: there is no transaction in progress
2026-08-06 16:18:16.859 UTC [28365] WARNING: there is no transaction in progress
2026-08-06 16:18:16.872 UTC [28365] WARNING: there is no transaction in progress
2026-08-06 16:18:16.889 UTC [28365] WARNING: there is no transaction in progress
2026-08-06 16:18:16.900 UTC [28365] WARNING: there is no transaction in progress
2026-08-06 16:18:16.908 UTC [28365] WARNING: there is no transaction in progress
2026-08-06 16:18:16.917 UTC [28365] WARNING: there is no transaction in progress
2026-08-06 16:18:16.927 UTC [28365] WARNING: there is no transaction in progress
2026-08-06 16:18:17,708 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 16:18:17,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 16:18:18,009 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 16:18:18,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 16:18:18,324 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 16:18:18,324 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 16:18:18,619 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 16:18:18,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 16:18:18,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 16:18:19,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-t3q3.c.gcp-upstream.internal';"
2026-08-06 16:18:19,478 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 16:18:19,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 16:18:19,781 [I] DB_PATCHES have already been applied
I20260806 16:18:19.788522 12246 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-admin:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskmdsaCi/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 16:18:19.788627 12246 mini_ranger.cc:193] Using host: 127.11.245.148
I20260806 16:18:19.795666 12246 mini_ranger.cc:227] Ranger admin URL: http://127.11.245.148:37681
Aug 06, 2026 4:18:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:18:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:18:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 4:18:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 4:18:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 4:18:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 4:18:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 16:18:21.214564 12246 mini_ranger.cc:160] Time spent starting Ranger: real 10.212s user 0.000s sys 0.017s
Aug 06, 2026 4:18:46 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 4:18:46 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 4:18:46 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 4:18:46 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 4:18:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 4:18:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 4:18:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 4:18:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 4:18:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 4:18:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 4:18:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 4:18:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 4:18:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 4:18:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 4:18:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 4:18:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 4:18:46 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 4:18:47 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 16:18:48.319371 12246 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.245.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.245.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.245.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/rangerkms_127.11.245.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.245.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/rangerkms_127.11.245.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.245.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.245.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.245.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/HTTP_127.11.245.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.245.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/HTTP_127.11.245.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 16:18:48 dist-test-slave-t3q3 krb5kdc[28054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033128, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 16:18:48.451295 12246 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 16:18:48.468878 12246 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 16:18:49.523713 12246 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 16:18:49.523860 12246 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-kms
I20260806 16:18:49.524012 12246 mini_ranger_kms.cc:227] Using postgres at 127.11.245.148:60545
2026-08-06 16:18:49,887 [I] DB FLAVOR :POSTGRES
2026-08-06 16:18:49,888 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 16:18:49,888 [I] Checking connection
2026-08-06 16:18:49,888 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 16:18:50,197 [I] connection success
2026-08-06 16:18:50,197 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 16:18:50,197 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 16:18:50,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 16:18:50,489 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 16:18:50,490 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 16:18:50,490 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 16:18:50,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:60545/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 16:18:50,796 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 16:18:51.396744 12246 mini_ranger_kms.cc:310] Created kms service
I20260806 16:18:51.663401 12246 mini_ranger_kms.cc:326] Created kudu user
I20260806 16:18:51.726584 12246 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 16:18:51 dist-test-slave-t3q3 krb5kdc[28054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033128, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
I20260806 16:18:52.222720 12246 mini_ranger_kms.cc:384] Added ranger policy
I20260806 16:18:52.222990 12246 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-kms:/tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 16:18:52.223030 12246 mini_ranger_kms.cc:240] Using host: 127.11.245.148
I20260806 16:18:52.229662 12246 mini_ranger_kms.cc:276] Ranger KMS PID: 28622
I20260806 16:18:52.229857 12246 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.245.148:40905
Aug 06, 2026 4:18:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:18:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:18:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 4:18:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 4:18:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 4:18:53 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 16:18:54 dist-test-slave-t3q3 krb5kdc[28054](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786033134, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.245.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:18:55.042598 12246 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.591s user 0.007s sys 0.314s
I20260806 16:18:55.042775 12246 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 16:18:55.042941 12246 mini_ranger_kms.cc:401] 127.11.245.148:40905/kms/v1/keys
Aug 06 16:18:55 dist-test-slave-t3q3 krb5kdc[28054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033134, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.245.148@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
Aug 06 16:18:55 dist-test-slave-t3q3 krb5kdc[28054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033134, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.245.148@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
Aug 06 16:18:58 dist-test-slave-t3q3 krb5kdc[28054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:18:58.580595 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:46377
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:37823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:46377
--ranger_config_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.245.148:40905/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:18:58.753355 28695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:18:58.753796 28695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:18:58.753873 28695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:18:58.758805 28695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:18:58.758917 28695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:18:58.758939 28695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:18:58.758956 28695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:18:58.758973 28695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:18:58.765975 28695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.245.148:40905/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:46377
--ranger_config_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:46377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:18:58.767630 28695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:18:58.769233 28695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:18:58.780660 28702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:58.780723 28695 server_base.cc:1034] running on GCE node
W20260806 16:18:58.780665 28704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:18:58.780660 28701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:18:58.782024 28695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:18:58.783629 28695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:18:58.784837 28695 hybrid_clock.cc:648] HybridClock initialized: now 1786033138784769 us; error 95 us; skew 500 ppm
Aug 06 16:18:58 dist-test-slave-t3q3 krb5kdc[28054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033138, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:18:58.788653 28695 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:18:58.790114 28695 webserver.cc:460] Webserver started at http://127.11.245.190:34317/ using document root <none> and password file <none>
I20260806 16:18:58.790967 28695 fs_manager.cc:359] Metadata directory not provided
I20260806 16:18:58.791051 28695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:18:58.791329 28695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:18:58 dist-test-slave-t3q3 krb5kdc[28054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033138, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
I20260806 16:18:58.852119 28695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "347793dfcfa2424eae1d136dbb790303"
format_stamp: "Formatted at 2026-08-06 16:18:58 on dist-test-slave-t3q3"
server_key: "48203986aaa0240cf930f7292fc7ec4a"
server_key_iv: "96013ab4bc9a3696077d5fbc05dfca63"
server_key_version: "kuduclusterkey@0"
I20260806 16:18:58.853060 28695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "347793dfcfa2424eae1d136dbb790303"
format_stamp: "Formatted at 2026-08-06 16:18:58 on dist-test-slave-t3q3"
server_key: "48203986aaa0240cf930f7292fc7ec4a"
server_key_iv: "96013ab4bc9a3696077d5fbc05dfca63"
server_key_version: "kuduclusterkey@0"
I20260806 16:18:58.858521 28695 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:18:58.881714 28715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:18:58.883699 28695 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:18:58.883956 28695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "347793dfcfa2424eae1d136dbb790303"
format_stamp: "Formatted at 2026-08-06 16:18:58 on dist-test-slave-t3q3"
server_key: "48203986aaa0240cf930f7292fc7ec4a"
server_key_iv: "96013ab4bc9a3696077d5fbc05dfca63"
server_key_version: "kuduclusterkey@0"
I20260806 16:18:58.884218 28695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:18:58.901741 28695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:18:58.905556 28695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:18:58.905829 28695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:18:58.916599 28695 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:46377
I20260806 16:18:58.916632 28790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:46377 every 8 connection(s)
I20260806 16:18:58.917963 28695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:18:58.922757 28791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:18:58.925082 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 28695
I20260806 16:18:58.925403 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:18:58.925824 12246 external_mini_cluster.cc:1428] Setting key 620a13ac808a0e26d31add0305edc660
I20260806 16:18:58.932368 28791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303: Bootstrap starting.
I20260806 16:18:58.936167 28791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303: Neither blocks nor log segments found. Creating new log.
Aug 06 16:18:58 dist-test-slave-t3q3 krb5kdc[28054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:18:58.937414 28791 log.cc:824] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303: Log is configured to *not* fsync() on all Append() calls
I20260806 16:18:58.940613 28791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303: No bootstrap required, opened a new log
I20260806 16:18:58.944635 28794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:18:58.927800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36544 (local address 127.11.245.190:46377)
0806 16:18:58.928428 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0806 16:18:58.928449 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:18:58.928499 (+ 50us) server_negotiation.cc:407] Connection header received
0806 16:18:58.929417 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:18:58.929441 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:18:58.929893 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:18:58.930354 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:18:58.931377 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:58.932667 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:58.933668 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:18:58.934304 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:18:58.934496 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:18:58.937587 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:18:58.937596 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:18:58.937626 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:18:58.937663 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:18:58.940718 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:18:58.941309 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:18:58.941316 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:18:58.941324 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:18:58.941479 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:18:58.941822 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:18:58.941825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:18:58.941827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:18:58.942419 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:18:58.942635 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 16:18:58.942999 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 16:18:58.943344 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":242,"threads_started":1}
I20260806 16:18:58.945232 28791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347793dfcfa2424eae1d136dbb790303" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46377 } }
I20260806 16:18:58.945562 28791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:18:58.945647 28791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 347793dfcfa2424eae1d136dbb790303, State: Initialized, Role: FOLLOWER
I20260806 16:18:58.946292 28791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347793dfcfa2424eae1d136dbb790303" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46377 } }
I20260806 16:18:58.946501 28791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:18:58.946597 28791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:18:58.946713 28791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:18:58.948267 28791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347793dfcfa2424eae1d136dbb790303" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46377 } }
I20260806 16:18:58.948701 28791 leader_election.cc:302] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 347793dfcfa2424eae1d136dbb790303; no voters:
I20260806 16:18:58.949110 28791 leader_election.cc:288] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:18:58.949256 28796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:18:58.949496 28796 raft_consensus.cc:686] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [term 1 LEADER]: Becoming Leader. State: Replica: 347793dfcfa2424eae1d136dbb790303, State: Running, Role: LEADER
I20260806 16:18:58.949913 28796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347793dfcfa2424eae1d136dbb790303" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46377 } }
I20260806 16:18:58.950709 28791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:18:58.951413 28796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 347793dfcfa2424eae1d136dbb790303. Latest consensus state: current_term: 1 leader_uuid: "347793dfcfa2424eae1d136dbb790303" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347793dfcfa2424eae1d136dbb790303" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46377 } } }
I20260806 16:18:58.951617 28796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [sys.catalog]: This master's current role is: LEADER
I20260806 16:18:58.952185 28802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:18:58.953482 28797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "347793dfcfa2424eae1d136dbb790303" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347793dfcfa2424eae1d136dbb790303" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46377 } } }
I20260806 16:18:58.953608 28797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303 [sys.catalog]: This master's current role is: LEADER
I20260806 16:18:58.956465 28802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:18:58.957026 28791 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 16:18:58.970594 28802 catalog_manager.cc:1347] Generated new cluster ID: ba7e7a8d97e446ffa4d816e1d417136f
I20260806 16:18:58.970796 28802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:18:58.990741 28802 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:18:58.992516 28802 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:18:59.005986 28802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303: Generated new TSK 0
I20260806 16:18:59.007519 28802 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:18:59.944801 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.245.148:40905/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:46377
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:37823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:19:00.159106 28852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:19:00.159696 28852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:19:00.159847 28852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:19:00.166960 28852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:19:00.167091 28852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:19:00.167205 28852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:19:00.177074 28852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.245.148:40905/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:46377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:19:00.180190 28852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:19:00.182185 28852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:19:00.197942 28861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:19:00.198108 28852 server_base.cc:1034] running on GCE node
W20260806 16:19:00.197943 28866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:00.197994 28863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:19:00.198984 28852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:19:00.199939 28852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:19:00.201283 28852 hybrid_clock.cc:648] HybridClock initialized: now 1786033140201256 us; error 47 us; skew 500 ppm
Aug 06 16:19:00 dist-test-slave-t3q3 krb5kdc[28054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033140, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:19:00.205415 28852 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:19:00.206673 28852 webserver.cc:460] Webserver started at http://127.11.245.129:42193/ using document root <none> and password file <none>
I20260806 16:19:00.207470 28852 fs_manager.cc:359] Metadata directory not provided
I20260806 16:19:00.207576 28852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:19:00.207872 28852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:19:00 dist-test-slave-t3q3 krb5kdc[28054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033140, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
I20260806 16:19:00.229087 28852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "573f4d3dd77f48a3926d5fb27ed3d62f"
format_stamp: "Formatted at 2026-08-06 16:19:00 on dist-test-slave-t3q3"
server_key: "29c806c95920b22a13064d4f4f3510a3"
server_key_iv: "5de5a7257b25397a934c05d19f83f85c"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:00.230146 28852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "573f4d3dd77f48a3926d5fb27ed3d62f"
format_stamp: "Formatted at 2026-08-06 16:19:00 on dist-test-slave-t3q3"
server_key: "29c806c95920b22a13064d4f4f3510a3"
server_key_iv: "5de5a7257b25397a934c05d19f83f85c"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:00.239902 28852 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.000s
I20260806 16:19:00.273368 28880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.001s user 0.000s sys 0.000s
I20260806 16:19:00.275902 28852 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.001s
I20260806 16:19:00.276252 28852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "573f4d3dd77f48a3926d5fb27ed3d62f"
format_stamp: "Formatted at 2026-08-06 16:19:00 on dist-test-slave-t3q3"
server_key: "29c806c95920b22a13064d4f4f3510a3"
server_key_iv: "5de5a7257b25397a934c05d19f83f85c"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:00.276592 28852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:19:00.301237 28852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:19:00.306275 28852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:19:00.306641 28852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:19:00.307686 28852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:19:00.309264 28852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:19:00.309394 28852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 16:19:00.309576 28852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:19:00.309677 28852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:00.327033 28852 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:41493
I20260806 16:19:00.328372 29045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:41493 every 8 connection(s)
I20260806 16:19:00.328866 28852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:19:00.334950 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 28852
I20260806 16:19:00.335125 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:19:00.335604 12246 external_mini_cluster.cc:1428] Setting key 03e22ce3730a9800392c6765651f3a89
Aug 06 16:19:00 dist-test-slave-t3q3 krb5kdc[28054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033140, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:19:00.350832 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:00.331489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:32929 (local address 127.11.245.190:46377)
0806 16:19:00.331807 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 16:19:00.331813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:19:00.332957 (+ 1144us) server_negotiation.cc:407] Connection header received
0806 16:19:00.334215 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:19:00.334219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:19:00.334300 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:19:00.334444 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:19:00.337261 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:00.338113 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:00.340195 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:00.340615 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:00.340725 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:00.344505 (+ 3780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:19:00.344512 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:19:00.344519 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:19:00.344559 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:19:00.347106 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:00.348044 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:00.348050 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:00.348052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:00.348208 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:00.348769 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:00.348775 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:00.348780 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:00.349145 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:19:00.349258 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 16:19:00.350339 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0806 16:19:00.350545 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:19:00.353942 29051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:00.332159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46377 (local address 127.11.245.129:32929)
0806 16:19:00.332757 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0806 16:19:00.332821 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 16:19:00.333939 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:19:00.334664 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:19:00.334678 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:19:00.335143 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:19:00.336451 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:00.336479 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:00.338357 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:00.338363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:00.339994 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:00.340011 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:00.341043 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:00.341049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:00.341306 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:00.342148 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:19:00.342179 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:19:00.344243 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:19:00.347307 (+ 3064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:19:00.347321 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:19:00.347356 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:19:00.347872 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:19:00.348353 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:19:00.348356 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:19:00.348358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:19:00.348635 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:19:00.349353 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:19:00.349366 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:19:00.349923 (+ 557us) client_negotiation.cc:769] Sending connection context
0806 16:19:00.351666 (+ 1743us) client_negotiation.cc:240] Negotiation successful
0806 16:19:00.352268 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":168,"threads_started":1}
I20260806 16:19:00.355736 29047 heartbeater.cc:347] Connected to a master server at 127.11.245.190:46377
I20260806 16:19:00.356122 29047 heartbeater.cc:464] Registering TS with master...
I20260806 16:19:00.356920 29047 heartbeater.cc:511] Master 127.11.245.190:46377 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
I20260806 16:19:00.361869 28735 ts_manager.cc:194] Registered new tserver with Master: 573f4d3dd77f48a3926d5fb27ed3d62f (127.11.245.129:41493)
I20260806 16:19:00.363442 28735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:32929
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:19:00.383111 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.245.148:40905/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:46377
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:37823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:19:00 dist-test-slave-t3q3 krb5kdc[28054](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786033140, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 16:19:00.576752 29056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:19:00.577267 29056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:19:00.577342 29056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:19:00.583775 29056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:19:00.583895 29056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:19:00.583995 29056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:19:00.593220 29056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.245.148:40905/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:46377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:19:00.595359 29056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:19:00.597132 29056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:19:00.613507 29069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:00.613894 29066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:00.613987 29065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:19:00.616784 29056 server_base.cc:1034] running on GCE node
I20260806 16:19:00.617789 29056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:19:00.618932 29056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:19:00.622248 29056 hybrid_clock.cc:648] HybridClock initialized: now 1786033140622153 us; error 104 us; skew 500 ppm
Aug 06 16:19:00 dist-test-slave-t3q3 krb5kdc[28054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033140, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:19:00.626984 29056 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:19:00.628587 29056 webserver.cc:460] Webserver started at http://127.11.245.130:42053/ using document root <none> and password file <none>
I20260806 16:19:00.629588 29056 fs_manager.cc:359] Metadata directory not provided
I20260806 16:19:00.629753 29056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:19:00.630120 29056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:19:00 dist-test-slave-t3q3 krb5kdc[28054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033140, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
I20260806 16:19:00.661989 29056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "84c03f79f5c8454b8457e18c2e78bd20"
format_stamp: "Formatted at 2026-08-06 16:19:00 on dist-test-slave-t3q3"
server_key: "85e689aa45c27031cc55c0048fa99711"
server_key_iv: "2a6b31cdb078c7fc2bf1b55841197001"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:00.663341 29056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "84c03f79f5c8454b8457e18c2e78bd20"
format_stamp: "Formatted at 2026-08-06 16:19:00 on dist-test-slave-t3q3"
server_key: "85e689aa45c27031cc55c0048fa99711"
server_key_iv: "2a6b31cdb078c7fc2bf1b55841197001"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:00.670090 29056 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260806 16:19:00.691244 29081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:00.693267 29056 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:19:00.693478 29056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "84c03f79f5c8454b8457e18c2e78bd20"
format_stamp: "Formatted at 2026-08-06 16:19:00 on dist-test-slave-t3q3"
server_key: "85e689aa45c27031cc55c0048fa99711"
server_key_iv: "2a6b31cdb078c7fc2bf1b55841197001"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:00.693683 29056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:19:00.718281 29056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:19:00.723505 29056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:19:00.723860 29056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:19:00.724936 29056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:19:00.726436 29056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:19:00.726573 29056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 16:19:00.726693 29056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:19:00.726763 29056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:00.744271 29056 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:42919
I20260806 16:19:00.746448 29056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:19:00.747191 29243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:42919 every 8 connection(s)
I20260806 16:19:00.754133 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 29056
I20260806 16:19:00.754344 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:19:00.754830 12246 external_mini_cluster.cc:1428] Setting key afcca3806fe85a1be67fea2ea583bd3b
Aug 06 16:19:00 dist-test-slave-t3q3 krb5kdc[28054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033140, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
I20260806 16:19:00.780089 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:00.750969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46377 (local address 127.11.245.130:42009)
0806 16:19:00.756833 (+ 5864us) negotiation.cc:107] Waiting for socket to connect
0806 16:19:00.756943 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0806 16:19:00.758435 (+ 1492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:19:00.759476 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:19:00.759489 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:19:00.759978 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:19:00.760900 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:00.760926 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:00.762224 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:00.762228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:00.763832 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:00.763845 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:00.764759 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:00.764763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:00.764997 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:00.770571 (+ 5574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:19:00.770604 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:19:00.772812 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:19:00.775582 (+ 2770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:19:00.775591 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:19:00.775620 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:19:00.776109 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:19:00.776508 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:19:00.776511 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:19:00.776513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:19:00.776735 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:19:00.777383 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:19:00.777393 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:19:00.777912 (+ 519us) client_negotiation.cc:769] Sending connection context
0806 16:19:00.778165 (+ 253us) client_negotiation.cc:240] Negotiation successful
0806 16:19:00.778633 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5460,"thread_start_us":253,"threads_started":1}
I20260806 16:19:00.780975 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:00.752777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:42009 (local address 127.11.245.190:46377)
0806 16:19:00.752987 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 16:19:00.752994 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:19:00.757081 (+ 4087us) server_negotiation.cc:407] Connection header received
0806 16:19:00.758757 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:19:00.758761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:19:00.758831 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:19:00.758890 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:19:00.761111 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:00.762032 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:00.763995 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:00.764395 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:00.764493 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:00.773237 (+ 8744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:19:00.773242 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:19:00.773246 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:19:00.773288 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:19:00.775389 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:00.776259 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:00.776264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:00.776267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:00.776387 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:00.776857 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:00.776860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:00.776862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:00.777167 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:19:00.777289 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 16:19:00.780587 (+ 3298us) server_negotiation.cc:299] Negotiation successful
0806 16:19:00.780782 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 16:19:00.783735 29244 heartbeater.cc:347] Connected to a master server at 127.11.245.190:46377
I20260806 16:19:00.784128 29244 heartbeater.cc:464] Registering TS with master...
I20260806 16:19:00.785014 29244 heartbeater.cc:511] Master 127.11.245.190:46377 requested a full tablet report, sending...
I20260806 16:19:00.786738 28735 ts_manager.cc:194] Registered new tserver with Master: 84c03f79f5c8454b8457e18c2e78bd20 (127.11.245.130:42919)
I20260806 16:19:00.787685 28735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:42009
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:19:00.820225 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.245.148:40905/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:46377
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:37823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:19:01 dist-test-slave-t3q3 krb5kdc[28054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033140, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
W20260806 16:19:01.052790 29250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:19:01.053402 29250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:19:01.053529 29250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:19:01.059842 29250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:19:01.060209 29250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:19:01.060393 29250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:19:01.069656 29250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.245.148:40905/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:46377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:19:01.072144 29250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:19:01.074082 29250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:19:01.093189 29250 server_base.cc:1034] running on GCE node
W20260806 16:19:01.093881 29263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:01.094273 29259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:01.094632 29260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:19:01.095556 29250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:19:01.096619 29250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:19:01.099907 29250 hybrid_clock.cc:648] HybridClock initialized: now 1786033141099815 us; error 98 us; skew 500 ppm
Aug 06 16:19:01 dist-test-slave-t3q3 krb5kdc[28054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033141, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:19:01.107342 29250 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:19:01.108966 29250 webserver.cc:460] Webserver started at http://127.11.245.131:44875/ using document root <none> and password file <none>
I20260806 16:19:01.109915 29250 fs_manager.cc:359] Metadata directory not provided
I20260806 16:19:01.110021 29250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:19:01.110334 29250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:19:01 dist-test-slave-t3q3 krb5kdc[28054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033141, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
I20260806 16:19:01.150575 29250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "1b803d6d57de447383a6bcd907d302bb"
format_stamp: "Formatted at 2026-08-06 16:19:01 on dist-test-slave-t3q3"
server_key: "53bacc61a3b37965d73bf9661fb52243"
server_key_iv: "766afa339d837ff3bbce43a86bdcdcb1"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:01.152100 29250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "1b803d6d57de447383a6bcd907d302bb"
format_stamp: "Formatted at 2026-08-06 16:19:01 on dist-test-slave-t3q3"
server_key: "53bacc61a3b37965d73bf9661fb52243"
server_key_iv: "766afa339d837ff3bbce43a86bdcdcb1"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:01.179888 29250 fs_manager.cc:691] Time spent creating directory manager: real 0.027s user 0.003s sys 0.002s
I20260806 16:19:01.203871 29276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:01.206315 29250 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260806 16:19:01.206523 29250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "1b803d6d57de447383a6bcd907d302bb"
format_stamp: "Formatted at 2026-08-06 16:19:01 on dist-test-slave-t3q3"
server_key: "53bacc61a3b37965d73bf9661fb52243"
server_key_iv: "766afa339d837ff3bbce43a86bdcdcb1"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:01.206728 29250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:19:01.259115 29250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:19:01.264024 29250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:19:01.264442 29250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:19:01.265503 29250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:19:01.267171 29250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:19:01.267316 29250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 16:19:01.267437 29250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:19:01.267509 29250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:01.283284 29250 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:37235
I20260806 16:19:01.285092 29250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:19:01.287302 29448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:37235 every 8 connection(s)
I20260806 16:19:01.288892 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 29250
I20260806 16:19:01.289065 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:19:01.289448 12246 external_mini_cluster.cc:1428] Setting key 7990e64b8999534ffd11d34c359f0869
Aug 06 16:19:01 dist-test-slave-t3q3 krb5kdc[28054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033141, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:19:01.312625 29454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.293004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:56313 (local address 127.11.245.190:46377)
0806 16:19:01.293364 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0806 16:19:01.293370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:19:01.297685 (+ 4315us) server_negotiation.cc:407] Connection header received
0806 16:19:01.297754 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:19:01.297757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:19:01.297834 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:19:01.297934 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:19:01.300439 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.301383 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.303279 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.303640 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.303723 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:01.307361 (+ 3638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:19:01.307366 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:19:01.307371 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:19:01.307409 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:19:01.309496 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:01.310297 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:01.310301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:01.310303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:01.310421 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:01.310899 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:01.310902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:01.310904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:01.311172 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:19:01.311274 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 16:19:01.312194 (+ 920us) server_negotiation.cc:299] Negotiation successful
0806 16:19:01.312373 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":125,"threads_started":1}
I20260806 16:19:01.317839 29453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.291636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46377 (local address 127.11.245.131:56313)
0806 16:19:01.295758 (+ 4122us) negotiation.cc:107] Waiting for socket to connect
0806 16:19:01.295846 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0806 16:19:01.297267 (+ 1421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:19:01.298371 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:19:01.298382 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:19:01.299203 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:19:01.300206 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:01.300235 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.301583 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.301587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:01.303136 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:01.303151 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.304002 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.304007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:01.304291 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:01.305186 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:19:01.305215 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:19:01.307138 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:19:01.309672 (+ 2534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:19:01.309680 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:19:01.309705 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:19:01.310158 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:19:01.310568 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:19:01.310571 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:19:01.310573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:19:01.310781 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:19:01.311355 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:19:01.311363 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:19:01.311807 (+ 444us) client_negotiation.cc:769] Sending connection context
0806 16:19:01.313210 (+ 1403us) client_negotiation.cc:240] Negotiation successful
0806 16:19:01.313718 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3766,"thread_start_us":200,"threads_started":1}
I20260806 16:19:01.319823 29449 heartbeater.cc:347] Connected to a master server at 127.11.245.190:46377
I20260806 16:19:01.320261 29449 heartbeater.cc:464] Registering TS with master...
I20260806 16:19:01.321199 29449 heartbeater.cc:511] Master 127.11.245.190:46377 requested a full tablet report, sending...
I20260806 16:19:01.322784 28735 ts_manager.cc:194] Registered new tserver with Master: 1b803d6d57de447383a6bcd907d302bb (127.11.245.131:37235)
I20260806 16:19:01.323760 28735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:56313
I20260806 16:19:01.329253 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:19:01.347476 29454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.332556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38984 (local address 127.11.245.190:46377)
0806 16:19:01.332745 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 16:19:01.332751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:19:01.336538 (+ 3787us) server_negotiation.cc:407] Connection header received
0806 16:19:01.336815 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:19:01.336819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:19:01.336887 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:19:01.336969 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:19:01.338360 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.339259 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.340594 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.340967 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.341048 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:01.343151 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:19:01.343156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:19:01.343159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:19:01.343190 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:19:01.345179 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:01.345738 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:01.345743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:01.345745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:01.345848 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:01.346227 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:01.346231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:01.346233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:01.346505 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:19:01.346634 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 16:19:01.347095 (+ 461us) server_negotiation.cc:299] Negotiation successful
0806 16:19:01.347261 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 16:19:01.353497 28735 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:19:01.360543 28735 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:19:01.367545 29047 heartbeater.cc:503] Master 127.11.245.190:46377 was elected leader, sending a full tablet report...
I20260806 16:19:01.400523 29468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.386547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:41493 (local address 127.0.0.1:44234)
0806 16:19:01.389939 (+ 3392us) negotiation.cc:107] Waiting for socket to connect
0806 16:19:01.389952 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:19:01.390047 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:19:01.390993 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:19:01.391001 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:19:01.391029 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:19:01.391216 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:01.391224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.393763 (+ 2539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.393767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:01.395638 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:01.395651 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.399736 (+ 4085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.399741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:01.399909 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:01.399981 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 16:19:01.400147 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 16:19:01.400323 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3327,"thread_start_us":99,"threads_started":1}
I20260806 16:19:01.401369 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.387044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44234 (local address 127.11.245.129:41493)
0806 16:19:01.388691 (+ 1647us) server_negotiation.cc:206] Beginning negotiation
0806 16:19:01.388701 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:19:01.390155 (+ 1454us) server_negotiation.cc:407] Connection header received
0806 16:19:01.390269 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:19:01.390275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:19:01.390567 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:19:01.390743 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:19:01.391363 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.393613 (+ 2250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.395792 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.397788 (+ 1996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.397933 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:01.398123 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 16:19:01.401047 (+ 2924us) server_negotiation.cc:299] Negotiation successful
0806 16:19:01.401213 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1492,"thread_start_us":143,"threads_started":1}
I20260806 16:19:01.411048 28946 tablet_service.cc:1467] Processing CreateTablet for tablet 747f059942a94621894f93c73f4651a1 (DEFAULT_TABLE table=test-table [id=34181e2e45c047909f2d00575fccd17b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:19:01.414997 28946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 747f059942a94621894f93c73f4651a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:19:01.434792 29472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.385815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:42919 (local address 127.0.0.1:42198)
0806 16:19:01.389637 (+ 3822us) negotiation.cc:107] Waiting for socket to connect
0806 16:19:01.389649 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:19:01.389743 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:19:01.403398 (+ 13655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:19:01.403403 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:19:01.403429 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:19:01.403598 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:01.403605 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.424652 (+ 21047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.424657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:01.426032 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:01.426049 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.434204 (+ 8155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.434209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:01.434346 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:01.434390 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 16:19:01.434449 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 16:19:01.434576 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3756,"thread_start_us":132,"threads_started":1}
I20260806 16:19:01.436329 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.386072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42198 (local address 127.11.245.130:42919)
0806 16:19:01.399026 (+ 12954us) server_negotiation.cc:206] Beginning negotiation
0806 16:19:01.399033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:19:01.399062 (+ 29us) server_negotiation.cc:407] Connection header received
0806 16:19:01.399150 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:19:01.399155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:19:01.399497 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:19:01.403948 (+ 4451us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:19:01.404207 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.405503 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.426275 (+ 20772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.427358 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.435585 (+ 8227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:01.435854 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 16:19:01.435939 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 16:19:01.436140 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12810,"thread_start_us":158,"threads_started":1}
I20260806 16:19:01.437203 29475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.383215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:37235 (local address 127.0.0.1:38638)
0806 16:19:01.389108 (+ 5893us) negotiation.cc:107] Waiting for socket to connect
0806 16:19:01.389185 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 16:19:01.389389 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:19:01.420415 (+ 31026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:19:01.420420 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:19:01.420466 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:19:01.420708 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:01.420718 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.429822 (+ 9104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.429827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:01.431124 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:01.431143 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.436713 (+ 5570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.436718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:01.436849 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:01.436884 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 16:19:01.436941 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 16:19:01.437043 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5653,"thread_start_us":140,"threads_started":1}
I20260806 16:19:01.438398 29469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.383393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38638 (local address 127.11.245.131:37235)
0806 16:19:01.398450 (+ 15057us) server_negotiation.cc:206] Beginning negotiation
0806 16:19:01.398465 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:19:01.398500 (+ 35us) server_negotiation.cc:407] Connection header received
0806 16:19:01.398610 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:19:01.398617 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:19:01.401937 (+ 3320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:19:01.420971 (+ 19034us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:19:01.421192 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.423891 (+ 2699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.431408 (+ 7517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.433993 (+ 2585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.437788 (+ 3795us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:01.438003 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 16:19:01.438080 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 16:19:01.438216 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14883,"thread_start_us":167,"threads_started":1}
I20260806 16:19:01.443819 29154 tablet_service.cc:1467] Processing CreateTablet for tablet 747f059942a94621894f93c73f4651a1 (DEFAULT_TABLE table=test-table [id=34181e2e45c047909f2d00575fccd17b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:19:01.445602 29154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 747f059942a94621894f93c73f4651a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:19:01.451953 29350 tablet_service.cc:1467] Processing CreateTablet for tablet 747f059942a94621894f93c73f4651a1 (DEFAULT_TABLE table=test-table [id=34181e2e45c047909f2d00575fccd17b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:19:01.453886 29350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 747f059942a94621894f93c73f4651a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:19:01.468789 29477 tablet_bootstrap.cc:492] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f: Bootstrap starting.
I20260806 16:19:01.470335 29479 tablet_bootstrap.cc:492] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20: Bootstrap starting.
I20260806 16:19:01.472229 29477 tablet_bootstrap.cc:654] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f: Neither blocks nor log segments found. Creating new log.
I20260806 16:19:01.472880 29478 tablet_bootstrap.cc:492] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb: Bootstrap starting.
I20260806 16:19:01.473059 29479 tablet_bootstrap.cc:654] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20: Neither blocks nor log segments found. Creating new log.
I20260806 16:19:01.475803 29477 log.cc:824] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f: Log is configured to *not* fsync() on all Append() calls
I20260806 16:19:01.478291 29478 tablet_bootstrap.cc:654] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb: Neither blocks nor log segments found. Creating new log.
I20260806 16:19:01.479411 29478 log.cc:824] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb: Log is configured to *not* fsync() on all Append() calls
I20260806 16:19:01.481504 29479 log.cc:824] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20: Log is configured to *not* fsync() on all Append() calls
I20260806 16:19:01.485450 29477 tablet_bootstrap.cc:492] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f: No bootstrap required, opened a new log
I20260806 16:19:01.485863 29477 ts_tablet_manager.cc:1397] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.003s
I20260806 16:19:01.491365 29477 raft_consensus.cc:348] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b803d6d57de447383a6bcd907d302bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 37235 } } peers { permanent_uuid: "84c03f79f5c8454b8457e18c2e78bd20" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42919 } } peers { permanent_uuid: "573f4d3dd77f48a3926d5fb27ed3d62f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41493 } }
I20260806 16:19:01.492175 29477 raft_consensus.cc:374] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:19:01.493096 29477 raft_consensus.cc:729] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 573f4d3dd77f48a3926d5fb27ed3d62f, State: Initialized, Role: FOLLOWER
I20260806 16:19:01.494334 29477 consensus_queue.cc:260] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b803d6d57de447383a6bcd907d302bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 37235 } } peers { permanent_uuid: "84c03f79f5c8454b8457e18c2e78bd20" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42919 } } peers { permanent_uuid: "573f4d3dd77f48a3926d5fb27ed3d62f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41493 } }
I20260806 16:19:01.497031 29478 tablet_bootstrap.cc:492] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb: No bootstrap required, opened a new log
I20260806 16:19:01.497426 29478 ts_tablet_manager.cc:1397] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb: Time spent bootstrapping tablet: real 0.025s user 0.008s sys 0.000s
I20260806 16:19:01.511106 29477 ts_tablet_manager.cc:1428] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f: Time spent starting tablet: real 0.025s user 0.006s sys 0.001s
I20260806 16:19:01.512033 29478 raft_consensus.cc:348] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b803d6d57de447383a6bcd907d302bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 37235 } } peers { permanent_uuid: "84c03f79f5c8454b8457e18c2e78bd20" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42919 } } peers { permanent_uuid: "573f4d3dd77f48a3926d5fb27ed3d62f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41493 } }
I20260806 16:19:01.512843 29478 raft_consensus.cc:374] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:19:01.513377 29478 raft_consensus.cc:729] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b803d6d57de447383a6bcd907d302bb, State: Initialized, Role: FOLLOWER
I20260806 16:19:01.520689 29478 consensus_queue.cc:260] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b803d6d57de447383a6bcd907d302bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 37235 } } peers { permanent_uuid: "84c03f79f5c8454b8457e18c2e78bd20" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42919 } } peers { permanent_uuid: "573f4d3dd77f48a3926d5fb27ed3d62f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41493 } }
I20260806 16:19:01.522284 29479 tablet_bootstrap.cc:492] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20: No bootstrap required, opened a new log
I20260806 16:19:01.522627 29479 ts_tablet_manager.cc:1397] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20: Time spent bootstrapping tablet: real 0.052s user 0.007s sys 0.000s
I20260806 16:19:01.522864 29478 ts_tablet_manager.cc:1428] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb: Time spent starting tablet: real 0.025s user 0.004s sys 0.004s
I20260806 16:19:01.526451 29449 heartbeater.cc:503] Master 127.11.245.190:46377 was elected leader, sending a full tablet report...
I20260806 16:19:01.535475 29479 raft_consensus.cc:348] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b803d6d57de447383a6bcd907d302bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 37235 } } peers { permanent_uuid: "84c03f79f5c8454b8457e18c2e78bd20" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42919 } } peers { permanent_uuid: "573f4d3dd77f48a3926d5fb27ed3d62f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41493 } }
I20260806 16:19:01.536114 29479 raft_consensus.cc:374] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:19:01.536257 29479 raft_consensus.cc:729] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c03f79f5c8454b8457e18c2e78bd20, State: Initialized, Role: FOLLOWER
I20260806 16:19:01.537055 29479 consensus_queue.cc:260] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b803d6d57de447383a6bcd907d302bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 37235 } } peers { permanent_uuid: "84c03f79f5c8454b8457e18c2e78bd20" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42919 } } peers { permanent_uuid: "573f4d3dd77f48a3926d5fb27ed3d62f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41493 } }
I20260806 16:19:01.538734 29479 ts_tablet_manager.cc:1428] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20: Time spent starting tablet: real 0.016s user 0.008s sys 0.000s
W20260806 16:19:01.539001 29451 tablet.cc:2369] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:19:01.539286 29244 heartbeater.cc:503] Master 127.11.245.190:46377 was elected leader, sending a full tablet report...
Aug 06 16:19:01 dist-test-slave-t3q3 krb5kdc[28054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033140, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
W20260806 16:19:01.585103 29049 tablet.cc:2369] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:19:01.687099 29483 raft_consensus.cc:482] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:19:01.687467 29483 raft_consensus.cc:504] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b803d6d57de447383a6bcd907d302bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 37235 } } peers { permanent_uuid: "84c03f79f5c8454b8457e18c2e78bd20" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42919 } } peers { permanent_uuid: "573f4d3dd77f48a3926d5fb27ed3d62f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41493 } }
I20260806 16:19:01.689517 29483 leader_election.cc:288] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b803d6d57de447383a6bcd907d302bb (127.11.245.131:37235), 84c03f79f5c8454b8457e18c2e78bd20 (127.11.245.130:42919)
I20260806 16:19:01.697294 29469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.691160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:51841 (local address 127.11.245.131:37235)
0806 16:19:01.691313 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 16:19:01.691318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:19:01.691824 (+ 506us) server_negotiation.cc:407] Connection header received
0806 16:19:01.692160 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:19:01.692165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:19:01.692235 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:19:01.692293 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:19:01.692859 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.693688 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.695716 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.696404 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.696504 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:01.696575 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 16:19:01.696985 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 16:19:01.697112 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 16:19:01.697693 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.690836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:34373 (local address 127.11.245.130:42919)
0806 16:19:01.691012 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 16:19:01.691018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:19:01.693211 (+ 2193us) server_negotiation.cc:407] Connection header received
0806 16:19:01.693417 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:19:01.693421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:19:01.693482 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:19:01.693542 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:19:01.694022 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.694815 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.696284 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.696930 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.697030 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:01.697082 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 16:19:01.697422 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 16:19:01.697544 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 16:19:01.698256 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.690061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:37235 (local address 127.11.245.129:51841)
0806 16:19:01.691701 (+ 1640us) negotiation.cc:107] Waiting for socket to connect
0806 16:19:01.691720 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:19:01.691945 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:19:01.692484 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:19:01.692487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:19:01.692511 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:19:01.692709 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:01.692717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.693861 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.693864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:01.695569 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:01.695581 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.696711 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.696715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:01.696820 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:01.696848 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 16:19:01.697871 (+ 1023us) client_negotiation.cc:240] Negotiation successful
0806 16:19:01.697988 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1550,"thread_start_us":150,"threads_started":1}
I20260806 16:19:01.698271 29490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.690641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:42919 (local address 127.11.245.129:34373)
0806 16:19:01.693087 (+ 2446us) negotiation.cc:107] Waiting for socket to connect
0806 16:19:01.693105 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:19:01.693289 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:19:01.693689 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:19:01.693693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:19:01.693713 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:19:01.693874 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:01.693882 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.694972 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.694975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:01.696132 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:01.696144 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.697179 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.697182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:01.697275 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:01.697295 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 16:19:01.698040 (+ 745us) client_negotiation.cc:240] Negotiation successful
0806 16:19:01.698112 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2355,"thread_start_us":120,"threads_started":1}
I20260806 16:19:01.699316 29381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "747f059942a94621894f93c73f4651a1" candidate_uuid: "573f4d3dd77f48a3926d5fb27ed3d62f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b803d6d57de447383a6bcd907d302bb" is_pre_election: true
I20260806 16:19:01.699303 29181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "747f059942a94621894f93c73f4651a1" candidate_uuid: "573f4d3dd77f48a3926d5fb27ed3d62f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c03f79f5c8454b8457e18c2e78bd20" is_pre_election: true
I20260806 16:19:01.699793 29381 raft_consensus.cc:2454] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 573f4d3dd77f48a3926d5fb27ed3d62f in term 0.
I20260806 16:19:01.699793 29181 raft_consensus.cc:2454] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 573f4d3dd77f48a3926d5fb27ed3d62f in term 0.
I20260806 16:19:01.700548 28881 leader_election.cc:302] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b803d6d57de447383a6bcd907d302bb, 573f4d3dd77f48a3926d5fb27ed3d62f; no voters:
I20260806 16:19:01.700985 29483 raft_consensus.cc:2781] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:19:01.701085 29483 raft_consensus.cc:482] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:19:01.701133 29483 raft_consensus.cc:3037] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:19:01.703142 29483 raft_consensus.cc:504] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b803d6d57de447383a6bcd907d302bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 37235 } } peers { permanent_uuid: "84c03f79f5c8454b8457e18c2e78bd20" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42919 } } peers { permanent_uuid: "573f4d3dd77f48a3926d5fb27ed3d62f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41493 } }
I20260806 16:19:01.703680 29483 leader_election.cc:288] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [CANDIDATE]: Term 1 election: Requested vote from peers 1b803d6d57de447383a6bcd907d302bb (127.11.245.131:37235), 84c03f79f5c8454b8457e18c2e78bd20 (127.11.245.130:42919)
I20260806 16:19:01.704389 29381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "747f059942a94621894f93c73f4651a1" candidate_uuid: "573f4d3dd77f48a3926d5fb27ed3d62f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b803d6d57de447383a6bcd907d302bb"
I20260806 16:19:01.704561 29381 raft_consensus.cc:3037] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:19:01.704641 29181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "747f059942a94621894f93c73f4651a1" candidate_uuid: "573f4d3dd77f48a3926d5fb27ed3d62f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c03f79f5c8454b8457e18c2e78bd20"
I20260806 16:19:01.704811 29181 raft_consensus.cc:3037] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:19:01.706380 29381 raft_consensus.cc:2454] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 573f4d3dd77f48a3926d5fb27ed3d62f in term 1.
I20260806 16:19:01.706615 29181 raft_consensus.cc:2454] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 573f4d3dd77f48a3926d5fb27ed3d62f in term 1.
I20260806 16:19:01.706830 28881 leader_election.cc:302] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b803d6d57de447383a6bcd907d302bb, 573f4d3dd77f48a3926d5fb27ed3d62f; no voters:
I20260806 16:19:01.707049 29483 raft_consensus.cc:2781] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:19:01.707355 29483 raft_consensus.cc:686] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [term 1 LEADER]: Becoming Leader. State: Replica: 573f4d3dd77f48a3926d5fb27ed3d62f, State: Running, Role: LEADER
I20260806 16:19:01.707862 29483 consensus_queue.cc:237] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b803d6d57de447383a6bcd907d302bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 37235 } } peers { permanent_uuid: "84c03f79f5c8454b8457e18c2e78bd20" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42919 } } peers { permanent_uuid: "573f4d3dd77f48a3926d5fb27ed3d62f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41493 } }
I20260806 16:19:01.712877 28735 catalog_manager.cc:5495] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f reported cstate change: term changed from 0 to 1, leader changed from <none> to 573f4d3dd77f48a3926d5fb27ed3d62f (127.11.245.129). New cstate: current_term: 1 leader_uuid: "573f4d3dd77f48a3926d5fb27ed3d62f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b803d6d57de447383a6bcd907d302bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 37235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c03f79f5c8454b8457e18c2e78bd20" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "573f4d3dd77f48a3926d5fb27ed3d62f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 41493 } health_report { overall_health: HEALTHY } } }
W20260806 16:19:01.748952 29245 tablet.cc:2369] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:19:01.765523 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.759762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44236 (local address 127.11.245.129:41493)
0806 16:19:01.759943 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 16:19:01.759949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:19:01.760233 (+ 284us) server_negotiation.cc:407] Connection header received
0806 16:19:01.760361 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:19:01.760365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:19:01.760471 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:19:01.760606 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:19:01.760978 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.761914 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.763284 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:01.763655 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:01.763774 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:01.764235 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:19:01.764987 (+ 752us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:19:01.765140 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 16:19:01.765224 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 16:19:01.765305 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 16:19:01.775514 29181 raft_consensus.cc:1261] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20 [term 1 FOLLOWER]: Refusing update from remote peer 573f4d3dd77f48a3926d5fb27ed3d62f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:19:01.776703 29483 consensus_queue.cc:1035] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c03f79f5c8454b8457e18c2e78bd20" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 42919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:19:01.788028 29381 raft_consensus.cc:1261] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb [term 1 FOLLOWER]: Refusing update from remote peer 573f4d3dd77f48a3926d5fb27ed3d62f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:19:01.789357 29483 consensus_queue.cc:1035] T 747f059942a94621894f93c73f4651a1 P 573f4d3dd77f48a3926d5fb27ed3d62f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b803d6d57de447383a6bcd907d302bb" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 37235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:19:01.806782 29495 mvcc.cc:205] Tried to move back new op lower bound from 7315591748700807168 to 7315591748441210880. Current Snapshot: MvccSnapshot[applied={T|T < 7315591748700807168}]
I20260806 16:19:01.822683 29496 mvcc.cc:205] Tried to move back new op lower bound from 7315591748700807168 to 7315591748441210880. Current Snapshot: MvccSnapshot[applied={T|T < 7315591748700807168}]
I20260806 16:19:01.823709 28735 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:19:01.823961 28735 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:19:01.827337 28735 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 347793dfcfa2424eae1d136dbb790303: Sending DeleteTablet for 3 replicas of tablet 747f059942a94621894f93c73f4651a1
I20260806 16:19:01.829353 29350 tablet_service.cc:1514] Processing DeleteTablet for tablet 747f059942a94621894f93c73f4651a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:19:01 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:38638
I20260806 16:19:01.829736 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 28852
I20260806 16:19:01.830054 29154 tablet_service.cc:1514] Processing DeleteTablet for tablet 747f059942a94621894f93c73f4651a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:19:01 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:42198
I20260806 16:19:01.834755 29511 tablet_replica.cc:330] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20: stopping tablet replica
I20260806 16:19:01.835312 29511 raft_consensus.cc:2229] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:19:01.835785 29511 raft_consensus.cc:2258] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:19:01.836411 29509 tablet_replica.cc:330] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb: stopping tablet replica
I20260806 16:19:01.837132 29509 raft_consensus.cc:2229] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:19:01.837713 29509 raft_consensus.cc:2258] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:19:01.843878 29511 ts_tablet_manager.cc:1905] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:19:01.848043 29509 ts_tablet_manager.cc:1905] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:19:01.854981 29511 ts_tablet_manager.cc:1918] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:19:01.856992 29509 ts_tablet_manager.cc:1918] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:19:01.857293 29511 log.cc:1196] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/747f059942a94621894f93c73f4651a1
I20260806 16:19:01.857872 29511 ts_tablet_manager.cc:1939] T 747f059942a94621894f93c73f4651a1 P 84c03f79f5c8454b8457e18c2e78bd20: Deleting consensus metadata
I20260806 16:19:01.858109 29509 log.cc:1196] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/747f059942a94621894f93c73f4651a1
W20260806 16:19:01.859639 28717 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:41493 (error 108)
I20260806 16:19:01.860334 29509 ts_tablet_manager.cc:1939] T 747f059942a94621894f93c73f4651a1 P 1b803d6d57de447383a6bcd907d302bb: Deleting consensus metadata
I20260806 16:19:01.862213 29475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.861723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:41493 (local address 127.0.0.1:44238)
0806 16:19:01.861997 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 16:19:01.862090 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
W20260806 16:19:01.862663 28717 catalog_manager.cc:4584] TS 573f4d3dd77f48a3926d5fb27ed3d62f (127.11.245.129:41493): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 747f059942a94621894f93c73f4651a1: Network error: Client connection negotiation failed: client connection to 127.11.245.129:41493: connect: Connection refused (error 111)
I20260806 16:19:01.862746 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 29056
I20260806 16:19:01.862949 28717 catalog_manager.cc:4841] TS 84c03f79f5c8454b8457e18c2e78bd20 (127.11.245.130:42919): tablet 747f059942a94621894f93c73f4651a1 (table test-table [id=34181e2e45c047909f2d00575fccd17b]) successfully deleted
I20260806 16:19:01.885046 28716 catalog_manager.cc:4841] TS 1b803d6d57de447383a6bcd907d302bb (127.11.245.131:37235): tablet 747f059942a94621894f93c73f4651a1 (table test-table [id=34181e2e45c047909f2d00575fccd17b]) successfully deleted
I20260806 16:19:01.899449 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 29250
I20260806 16:19:01.920153 29475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.917267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:41493 (local address 127.0.0.1:44248)
0806 16:19:01.919880 (+ 2613us) negotiation.cc:107] Waiting for socket to connect
0806 16:19:01.920002 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2459}
I20260806 16:19:01.921716 29475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:01.921336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:41493 (local address 127.0.0.1:44264)
0806 16:19:01.921544 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 16:19:01.921610 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260806 16:19:01.932305 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 28695
I20260806 16:19:01.952710 12246 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 16:19:02.312268 12246 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 16:19:02.312572 12246 mini_ranger.cc:66] Stopping Ranger...
I20260806 16:19:02.681134 12246 mini_ranger.cc:68] Stopped Ranger
2026-08-06 16:19:02.681 UTC [28101] LOG: received smart shutdown request
2026-08-06 16:19:02.686 UTC [28101] LOG: background worker "logical replication launcher" (PID 28111) exited with exit code 1
2026-08-06 16:19:02.687 UTC [28106] LOG: shutting down
2026-08-06 16:19:02.901 UTC [28101] LOG: database system is shut down
2026-08-06T16:19:02Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (57747 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:19:03 dist-test-slave-t3q3 krb5kdc[29532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:19:03 dist-test-slave-t3q3 krb5kdc[29532](info): set up 2 sockets
Aug 06 16:19:03 dist-test-slave-t3q3 krb5kdc[29532](info): commencing operation
krb5kdc: starting...
W20260806 16:19:04.359092 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:19:04 dist-test-slave-t3q3 krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:19:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:19:04Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.245.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.245.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.245.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/rangeradmin_127.11.245.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.245.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/rangeradmin_127.11.245.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.245.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.245.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.245.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/rangerlookup_127.11.245.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.245.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/rangerlookup_127.11.245.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.245.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.245.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.245.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/HTTP_127.11.245.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.245.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/HTTP_127.11.245.148@KRBTEST.COM.keytab.
I20260806 16:19:04.592113 12246 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskmdsaCi/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/postgres -l logfile start
2026-08-06 16:19:07.625 UTC [29574] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 16:19:07.625 UTC [29574] LOG: listening on IPv4 address "127.11.245.148", port 35597
2026-08-06 16:19:07.630 UTC [29574] LOG: listening on Unix socket "/tmp/.s.PGSQL.35597"
2026-08-06 16:19:07.657 UTC [29580] LOG: database system was shut down at 2026-08-06 16:19:05 UTC
2026-08-06 16:19:07.662 UTC [29574] LOG: database system is ready to accept connections
I20260806 16:19:08.852757 12246 mini_postgres.cc:96] Postgres bound to 35597
2026-08-06 16:19:08.864 UTC [29594] FATAL: database "slave" does not exist
127.11.245.148:35597 - accepting connections
I20260806 16:19:08.865574 12246 mini_ranger.cc:161] Starting Ranger...
I20260806 16:19:08.886252 12246 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 16:19:09.462620 12246 mini_ranger.cc:87] Created ranger Postgres database
I20260806 16:19:09.462754 12246 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-admin
2026-08-06 16:19:09,792 [I] DB FLAVOR :POSTGRES
2026-08-06 16:19:09,793 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 16:19:09,793 [I] Checking connection..
2026-08-06 16:19:09,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 16:19:10,106 [I] Checking connection passed.
2026-08-06 16:19:10,107 [I] --------- Verifying version history table ---------
2026-08-06 16:19:10,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:19:10,417 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 16:19:10,417 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:19:10,737 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 16:19:10,738 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 16:19:10,738 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 16:19:11.039 UTC [29692] WARNING: there is no transaction in progress
2026-08-06 16:19:11,052 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 16:19:11,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:19:11,351 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 16:19:11,352 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 16:19:11,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 16:19:11,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 16:19:11,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-t3q3.c.gcp-upstream.internal','N') ;"
2026-08-06 16:19:12,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 16:19:12,541 [I] Table x_portal_user does not exist in database ranger
2026-08-06 16:19:12,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 16:19:12,857 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 16:19:12,858 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 16:19:13,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 16:19:13,493 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 16:19:13,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 16:19:14.746 UTC [29833] WARNING: there is no transaction in progress
2026-08-06 16:19:14.763 UTC [29833] WARNING: there is no transaction in progress
2026-08-06 16:19:14.785 UTC [29833] WARNING: there is no transaction in progress
2026-08-06 16:19:14.804 UTC [29833] WARNING: there is no transaction in progress
2026-08-06 16:19:14.827 UTC [29833] WARNING: there is no transaction in progress
2026-08-06 16:19:14.851 UTC [29833] WARNING: there is no transaction in progress
2026-08-06 16:19:14.925 UTC [29833] WARNING: there is no transaction in progress
2026-08-06 16:19:14.933 UTC [29833] WARNING: there is no transaction in progress
2026-08-06 16:19:14.950 UTC [29833] WARNING: there is no transaction in progress
2026-08-06 16:19:14.963 UTC [29833] WARNING: there is no transaction in progress
2026-08-06 16:19:14.977 UTC [29833] WARNING: there is no transaction in progress
2026-08-06 16:19:14.993 UTC [29833] WARNING: there is no transaction in progress
2026-08-06 16:19:15.008 UTC [29833] WARNING: there is no transaction in progress
2026-08-06 16:19:15.016 UTC [29833] WARNING: there is no transaction in progress
2026-08-06 16:19:15.024 UTC [29833] WARNING: there is no transaction in progress
2026-08-06 16:19:15.033 UTC [29833] WARNING: there is no transaction in progress
2026-08-06 16:19:15,865 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 16:19:15,865 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 16:19:16,145 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 16:19:16,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 16:19:16,461 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 16:19:16,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 16:19:16,802 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 16:19:16,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 16:19:17,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 16:19:17,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-t3q3.c.gcp-upstream.internal';"
2026-08-06 16:19:17,679 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 16:19:17,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 16:19:17,995 [I] DB_PATCHES have already been applied
I20260806 16:19:18.003324 12246 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-admin:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskmdsaCi/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 16:19:18.003432 12246 mini_ranger.cc:193] Using host: 127.11.245.148
I20260806 16:19:18.010222 12246 mini_ranger.cc:227] Ranger admin URL: http://127.11.245.148:38757
Aug 06, 2026 4:19:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:19:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:19:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 4:19:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 4:19:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 4:19:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 4:19:19 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 16:19:19.442440 12246 mini_ranger.cc:160] Time spent starting Ranger: real 10.577s user 0.000s sys 0.017s
Aug 06, 2026 4:19:43 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 4:19:43 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 4:19:43 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 4:19:43 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 4:19:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 4:19:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 4:19:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 4:19:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 4:19:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 4:19:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 4:19:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 4:19:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 4:19:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 4:19:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 4:19:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 4:19:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 4:19:43 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 4:19:45 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 16:19:45.967223 12246 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.245.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.245.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.245.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/rangerkms_127.11.245.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.245.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/rangerkms_127.11.245.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.245.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.245.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.245.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/HTTP_127.11.245.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.245.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/HTTP_127.11.245.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 16:19:46 dist-test-slave-t3q3 krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033186, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 16:19:46.093775 12246 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 16:19:46.111541 12246 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 16:19:47.094487 12246 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 16:19:47.094594 12246 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-kms
I20260806 16:19:47.094609 12246 mini_ranger_kms.cc:227] Using postgres at 127.11.245.148:35597
2026-08-06 16:19:47,340 [I] DB FLAVOR :POSTGRES
2026-08-06 16:19:47,340 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 16:19:47,340 [I] Checking connection
2026-08-06 16:19:47,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 16:19:47,657 [I] connection success
2026-08-06 16:19:47,657 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 16:19:47,657 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 16:19:47,657 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 16:19:47,953 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 16:19:47,953 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 16:19:47,953 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 16:19:47,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.245.148:35597/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 16:19:48,303 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 16:19:48.899588 12246 mini_ranger_kms.cc:310] Created kms service
I20260806 16:19:49.164711 12246 mini_ranger_kms.cc:326] Created kudu user
I20260806 16:19:49.228363 12246 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 16:19:49 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033186, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
I20260806 16:19:49.786644 12246 mini_ranger_kms.cc:384] Added ranger policy
I20260806 16:19:49.786912 12246 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-kms:/tmp/dist-test-taskmdsaCi/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskmdsaCi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 16:19:49.786949 12246 mini_ranger_kms.cc:240] Using host: 127.11.245.148
I20260806 16:19:49.794252 12246 mini_ranger_kms.cc:276] Ranger KMS PID: 30077
I20260806 16:19:49.794386 12246 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.245.148:44529
Aug 06, 2026 4:19:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:19:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:19:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 4:19:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 4:19:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 4:19:51 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 16:19:52 dist-test-slave-t3q3 krb5kdc[29532](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786033192, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.245.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:19:52.727634 12246 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.634s user 0.000s sys 0.227s
I20260806 16:19:52.727852 12246 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 16:19:52.727969 12246 mini_ranger_kms.cc:401] 127.11.245.148:44529/kms/v1/keys
Aug 06 16:19:52 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033192, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.245.148@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
Aug 06 16:19:53 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033192, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.245.148@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
Aug 06 16:19:56 dist-test-slave-t3q3 krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:19:56.176038 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:33055
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:33055
--ranger_config_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.245.148:44529/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:19:56.322196 30151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:19:56.322571 30151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:19:56.322618 30151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:19:56.327033 30151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:19:56.327113 30151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:19:56.327135 30151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:19:56.327157 30151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:19:56.327172 30151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:19:56.333588 30151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.245.148:44529/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:33055
--ranger_config_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:33055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:19:56.335140 30151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:19:56.336647 30151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:19:56.347993 30151 server_base.cc:1034] running on GCE node
W20260806 16:19:56.347924 30156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:56.347924 30162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:56.347931 30157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:19:56.349077 30151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:19:56.350533 30151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:19:56.351740 30151 hybrid_clock.cc:648] HybridClock initialized: now 1786033196351709 us; error 51 us; skew 500 ppm
Aug 06 16:19:56 dist-test-slave-t3q3 krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033196, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:19:56.355448 30151 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:19:56.356729 30151 webserver.cc:460] Webserver started at http://127.11.245.190:37737/ using document root <none> and password file <none>
I20260806 16:19:56.357414 30151 fs_manager.cc:359] Metadata directory not provided
I20260806 16:19:56.357470 30151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:19:56.357685 30151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:19:56 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033196, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
I20260806 16:19:56.432976 30151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "96580ebc0f934b2db6991a8a0db698b9"
format_stamp: "Formatted at 2026-08-06 16:19:56 on dist-test-slave-t3q3"
server_key: "f4d7f6d8fc009b01177fac1bcc2e309e"
server_key_iv: "a1d08312316394f377304951df3f15fd"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:56.433893 30151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "96580ebc0f934b2db6991a8a0db698b9"
format_stamp: "Formatted at 2026-08-06 16:19:56 on dist-test-slave-t3q3"
server_key: "f4d7f6d8fc009b01177fac1bcc2e309e"
server_key_iv: "a1d08312316394f377304951df3f15fd"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:56.439662 30151 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 16:19:56.461411 30170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:56.463402 30151 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 16:19:56.463619 30151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "96580ebc0f934b2db6991a8a0db698b9"
format_stamp: "Formatted at 2026-08-06 16:19:56 on dist-test-slave-t3q3"
server_key: "f4d7f6d8fc009b01177fac1bcc2e309e"
server_key_iv: "a1d08312316394f377304951df3f15fd"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:56.463840 30151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:19:56.492036 30151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:19:56.495927 30151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:19:56.496228 30151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:19:56.506080 30151 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:33055
I20260806 16:19:56.506080 30237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:33055 every 8 connection(s)
I20260806 16:19:56.507396 30151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:19:56.509404 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 30151
I20260806 16:19:56.509593 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:19:56.509968 12246 external_mini_cluster.cc:1428] Setting key defddcf2d62ab12b3d558631e6041ab4
I20260806 16:19:56.511749 30239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:19:56.520716 30239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9: Bootstrap starting.
Aug 06 16:19:56 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:19:56.524818 30239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9: Neither blocks nor log segments found. Creating new log.
I20260806 16:19:56.526615 30239 log.cc:824] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9: Log is configured to *not* fsync() on all Append() calls
I20260806 16:19:56.528154 30243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:56.511694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60672 (local address 127.11.245.190:33055)
0806 16:19:56.512271 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0806 16:19:56.512294 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 16:19:56.512347 (+ 53us) server_negotiation.cc:407] Connection header received
0806 16:19:56.513377 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:19:56.513407 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:19:56.513896 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:19:56.514329 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:19:56.515122 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:56.516766 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:56.517782 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:56.518310 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:56.518547 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:56.521887 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:19:56.521895 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:19:56.521928 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:19:56.521971 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:19:56.524212 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:56.524775 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:56.524783 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:56.524791 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:56.525024 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:56.525349 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:56.525355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:56.525358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:56.526069 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:19:56.526271 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 16:19:56.526621 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 16:19:56.526988 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":152,"threads_started":1}
I20260806 16:19:56.529464 30239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9: No bootstrap required, opened a new log
I20260806 16:19:56.533434 30239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96580ebc0f934b2db6991a8a0db698b9" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33055 } }
I20260806 16:19:56.533728 30239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:19:56.533766 30239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96580ebc0f934b2db6991a8a0db698b9, State: Initialized, Role: FOLLOWER
I20260806 16:19:56.534308 30239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96580ebc0f934b2db6991a8a0db698b9" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33055 } }
I20260806 16:19:56.534466 30239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:19:56.534559 30239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:19:56.534694 30239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:19:56.535873 30239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96580ebc0f934b2db6991a8a0db698b9" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33055 } }
I20260806 16:19:56.536331 30239 leader_election.cc:302] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96580ebc0f934b2db6991a8a0db698b9; no voters:
I20260806 16:19:56.536816 30239 leader_election.cc:288] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:19:56.536958 30245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:19:56.537250 30245 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 1 LEADER]: Becoming Leader. State: Replica: 96580ebc0f934b2db6991a8a0db698b9, State: Running, Role: LEADER
I20260806 16:19:56.537673 30245 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96580ebc0f934b2db6991a8a0db698b9" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33055 } }
I20260806 16:19:56.538384 30239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:19:56.539819 30246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96580ebc0f934b2db6991a8a0db698b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96580ebc0f934b2db6991a8a0db698b9" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33055 } } }
I20260806 16:19:56.539826 30247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96580ebc0f934b2db6991a8a0db698b9. Latest consensus state: current_term: 1 leader_uuid: "96580ebc0f934b2db6991a8a0db698b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96580ebc0f934b2db6991a8a0db698b9" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33055 } } }
I20260806 16:19:56.539999 30247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [sys.catalog]: This master's current role is: LEADER
I20260806 16:19:56.539999 30246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [sys.catalog]: This master's current role is: LEADER
I20260806 16:19:56.541236 30239 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 16:19:56.541445 30256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:19:56.549549 30256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:19:56.558792 30256 catalog_manager.cc:1347] Generated new cluster ID: 242cf77fab1443a683065feb7ae13a1e
I20260806 16:19:56.558890 30256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:19:56.580042 30256 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:19:56.581949 30256 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:19:56.588562 30256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9: Generated new TSK 0
I20260806 16:19:56.589769 30256 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:19:57.367928 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.245.148:44529/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33055
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:19:57.537322 30295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:19:57.537704 30295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:19:57.537829 30295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:19:57.542768 30295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:19:57.542879 30295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:19:57.542994 30295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:19:57.549428 30295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.245.148:44529/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:19:57.550979 30295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:19:57.552489 30295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:19:57.568109 30306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:57.568349 30308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:57.568943 30305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:19:57.570024 30295 server_base.cc:1034] running on GCE node
I20260806 16:19:57.570715 30295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:19:57.571756 30295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:19:57.573083 30295 hybrid_clock.cc:648] HybridClock initialized: now 1786033197573057 us; error 43 us; skew 500 ppm
Aug 06 16:19:57 dist-test-slave-t3q3 krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033197, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:19:57.576725 30295 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:19:57.577956 30295 webserver.cc:460] Webserver started at http://127.11.245.129:40617/ using document root <none> and password file <none>
I20260806 16:19:57.578770 30295 fs_manager.cc:359] Metadata directory not provided
I20260806 16:19:57.578843 30295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:19:57.579118 30295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:19:57 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033197, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
I20260806 16:19:57.609668 30295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "84783f594c914ad8ab166b8a0ac6daf7"
format_stamp: "Formatted at 2026-08-06 16:19:57 on dist-test-slave-t3q3"
server_key: "ac592f36ac0098cdd57f7bf89bbfa302"
server_key_iv: "e5ff89b41ba27cbd56f915a4c006c477"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:57.610903 30295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "84783f594c914ad8ab166b8a0ac6daf7"
format_stamp: "Formatted at 2026-08-06 16:19:57 on dist-test-slave-t3q3"
server_key: "ac592f36ac0098cdd57f7bf89bbfa302"
server_key_iv: "e5ff89b41ba27cbd56f915a4c006c477"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:57.617667 30295 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:19:57.648730 30322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:57.651891 30295 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.002s sys 0.001s
I20260806 16:19:57.652173 30295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "84783f594c914ad8ab166b8a0ac6daf7"
format_stamp: "Formatted at 2026-08-06 16:19:57 on dist-test-slave-t3q3"
server_key: "ac592f36ac0098cdd57f7bf89bbfa302"
server_key_iv: "e5ff89b41ba27cbd56f915a4c006c477"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:57.652483 30295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:19:57.669869 30295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:19:57.674934 30295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:19:57.675199 30295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:19:57.676250 30295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:19:57.677719 30295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:19:57.677794 30295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:57.677863 30295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:19:57.677901 30295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:57.694049 30295 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:43119
I20260806 16:19:57.695808 30295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:19:57.698261 30480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:43119 every 8 connection(s)
I20260806 16:19:57.701432 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 30295
I20260806 16:19:57.701598 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:19:57.702023 12246 external_mini_cluster.cc:1428] Setting key 8673051c862ab2e7ff5551d2b1958928
Aug 06 16:19:57 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033197, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:19:57.723953 30484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:57.703322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:52019 (local address 127.11.245.190:33055)
0806 16:19:57.708297 (+ 4975us) server_negotiation.cc:206] Beginning negotiation
0806 16:19:57.708303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:19:57.708321 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:19:57.708709 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:19:57.708713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:19:57.708781 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:19:57.708881 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:19:57.711048 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:57.712003 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:57.714008 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:57.714383 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:57.714485 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:57.718475 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:19:57.718480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:19:57.718484 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:19:57.718520 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:19:57.720724 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:57.721549 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:57.721556 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:57.721560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:57.721667 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:57.722218 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:57.722222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:57.722224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:57.722540 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:19:57.722623 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 16:19:57.723530 (+ 907us) server_negotiation.cc:299] Negotiation successful
0806 16:19:57.723702 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4867,"thread_start_us":95,"threads_started":1}
I20260806 16:19:57.726456 30483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:57.701747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:33055 (local address 127.11.245.129:52019)
0806 16:19:57.706239 (+ 4492us) negotiation.cc:107] Waiting for socket to connect
0806 16:19:57.706329 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0806 16:19:57.707853 (+ 1524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:19:57.709302 (+ 1449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:19:57.709314 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:19:57.709842 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:19:57.710811 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:57.710837 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:57.712240 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:57.712244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:57.713817 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:57.713831 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:57.714776 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:57.714781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:57.715043 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:57.715943 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:19:57.715970 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:19:57.718205 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:19:57.720922 (+ 2717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:19:57.720940 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:19:57.720967 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:19:57.721411 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:19:57.721802 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:19:57.721806 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:19:57.721808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:19:57.722079 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:19:57.722719 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:19:57.722727 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:19:57.723203 (+ 476us) client_negotiation.cc:769] Sending connection context
0806 16:19:57.724327 (+ 1124us) client_negotiation.cc:240] Negotiation successful
0806 16:19:57.724793 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4140,"thread_start_us":188,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
I20260806 16:19:57.728699 30481 heartbeater.cc:347] Connected to a master server at 127.11.245.190:33055
I20260806 16:19:57.729089 30481 heartbeater.cc:464] Registering TS with master...
I20260806 16:19:57.729979 30481 heartbeater.cc:511] Master 127.11.245.190:33055 requested a full tablet report, sending...
I20260806 16:19:57.732779 30193 ts_manager.cc:194] Registered new tserver with Master: 84783f594c914ad8ab166b8a0ac6daf7 (127.11.245.129:43119)
I20260806 16:19:57.734807 30193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:52019
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:19:57.754235 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.245.148:44529/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33055
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:19:57 dist-test-slave-t3q3 krb5kdc[29532](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786033197, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 16:19:57.975622 30488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:19:57.976215 30488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:19:57.976316 30488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:19:57.983968 30488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:19:57.984108 30488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:19:57.984210 30488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:19:57.993979 30488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.245.148:44529/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:19:57.996232 30488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:19:57.998298 30488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:19:58.016117 30501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:58.016197 30499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:58.016665 30496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:19:58.017355 30488 server_base.cc:1034] running on GCE node
I20260806 16:19:58.018013 30488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:19:58.019121 30488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:19:58.020339 30488 hybrid_clock.cc:648] HybridClock initialized: now 1786033198020312 us; error 43 us; skew 500 ppm
Aug 06 16:19:58 dist-test-slave-t3q3 krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033198, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:19:58.025229 30488 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:19:58.026715 30488 webserver.cc:460] Webserver started at http://127.11.245.130:34917/ using document root <none> and password file <none>
I20260806 16:19:58.027676 30488 fs_manager.cc:359] Metadata directory not provided
I20260806 16:19:58.027782 30488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:19:58.028126 30488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:19:58 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033198, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
I20260806 16:19:58.051515 30488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "cf996ba5c1d0485c9b59dd83e01baab4"
format_stamp: "Formatted at 2026-08-06 16:19:58 on dist-test-slave-t3q3"
server_key: "74d82c08206358f10d4dc01548035d38"
server_key_iv: "9b4cbfa59427abfb61d338b15fa6f106"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:58.052627 30488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "cf996ba5c1d0485c9b59dd83e01baab4"
format_stamp: "Formatted at 2026-08-06 16:19:58 on dist-test-slave-t3q3"
server_key: "74d82c08206358f10d4dc01548035d38"
server_key_iv: "9b4cbfa59427abfb61d338b15fa6f106"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:58.066584 30488 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.002s sys 0.004s
I20260806 16:19:58.098675 30512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:58.101177 30488 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.000s
I20260806 16:19:58.101418 30488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "cf996ba5c1d0485c9b59dd83e01baab4"
format_stamp: "Formatted at 2026-08-06 16:19:58 on dist-test-slave-t3q3"
server_key: "74d82c08206358f10d4dc01548035d38"
server_key_iv: "9b4cbfa59427abfb61d338b15fa6f106"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:58.101620 30488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:19:58.139714 30488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:19:58.148837 30488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:19:58.149138 30488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:19:58.150113 30488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:19:58.151511 30488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:19:58.151587 30488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:58.151659 30488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:19:58.151692 30488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:58.165946 30488 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:36601
I20260806 16:19:58.166004 30672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:36601 every 8 connection(s)
I20260806 16:19:58.167543 30488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:19:58.176613 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 30488
I20260806 16:19:58.176782 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:19:58.177202 12246 external_mini_cluster.cc:1428] Setting key 5ef206220a4972db2767ea3f62297712
Aug 06 16:19:58 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033198, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:19:58.189359 30484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:58.170893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:48813 (local address 127.11.245.190:33055)
0806 16:19:58.171105 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 16:19:58.171111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:19:58.172500 (+ 1389us) server_negotiation.cc:407] Connection header received
0806 16:19:58.174066 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:19:58.174070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:19:58.174153 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:19:58.174278 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:19:58.175844 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:58.176754 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:58.179490 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:58.179844 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:58.179949 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:58.183882 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:19:58.183890 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:19:58.183896 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:19:58.183946 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:19:58.186085 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:58.186889 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:58.186893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:58.186895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:58.187007 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:58.187485 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:58.187489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:58.187491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:58.187846 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:19:58.187948 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 16:19:58.188897 (+ 949us) server_negotiation.cc:299] Negotiation successful
0806 16:19:58.189102 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 16:19:58.191931 30676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:58.171515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:33055 (local address 127.11.245.130:48813)
0806 16:19:58.172266 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0806 16:19:58.172338 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 16:19:58.173639 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:19:58.174515 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:19:58.174524 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:19:58.174940 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:19:58.175626 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:58.175653 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:58.176984 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:58.176988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:58.179337 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:58.179351 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:58.180269 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:58.180275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:58.180524 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:58.181408 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:19:58.181437 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:19:58.183635 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:19:58.186245 (+ 2610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:19:58.186257 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:19:58.186285 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:19:58.186757 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:19:58.187131 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:19:58.187134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:19:58.187136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:19:58.187354 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:19:58.188045 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:19:58.188082 (+ 37us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:19:58.188547 (+ 465us) client_negotiation.cc:769] Sending connection context
0806 16:19:58.189943 (+ 1396us) client_negotiation.cc:240] Negotiation successful
0806 16:19:58.190421 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":195,"threads_started":1}
I20260806 16:19:58.194350 30673 heartbeater.cc:347] Connected to a master server at 127.11.245.190:33055
I20260806 16:19:58.194702 30673 heartbeater.cc:464] Registering TS with master...
I20260806 16:19:58.195542 30673 heartbeater.cc:511] Master 127.11.245.190:33055 requested a full tablet report, sending...
I20260806 16:19:58.197220 30193 ts_manager.cc:194] Registered new tserver with Master: cf996ba5c1d0485c9b59dd83e01baab4 (127.11.245.130:36601)
I20260806 16:19:58.198487 30193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:48813
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:19:58.245801 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.245.148:44529/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33055
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:19:58 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033197, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
W20260806 16:19:58.471069 30680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:19:58.471627 30680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:19:58.471736 30680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:19:58.479231 30680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:19:58.479365 30680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:19:58.479470 30680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:19:58.491925 30680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.245.148:44529/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:19:58.494275 30680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:19:58.496163 30680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:19:58.521690 30690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:58.521819 30693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:58.521697 30687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:19:58.522671 30680 server_base.cc:1034] running on GCE node
I20260806 16:19:58.523325 30680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:19:58.524451 30680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:19:58.527791 30680 hybrid_clock.cc:648] HybridClock initialized: now 1786033198527742 us; error 62 us; skew 500 ppm
Aug 06 16:19:58 dist-test-slave-t3q3 krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033198, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:19:58.533062 30680 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:19:58.534770 30680 webserver.cc:460] Webserver started at http://127.11.245.131:46645/ using document root <none> and password file <none>
I20260806 16:19:58.535756 30680 fs_manager.cc:359] Metadata directory not provided
I20260806 16:19:58.535851 30680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:19:58.536253 30680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:19:58 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033198, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
I20260806 16:19:58.568732 30680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "b2e3251de3c044a98343877d8325a469"
format_stamp: "Formatted at 2026-08-06 16:19:58 on dist-test-slave-t3q3"
server_key: "45b3add53aa6a97ef3bbe31fa8e943de"
server_key_iv: "be56bdc1c38b2760ca1d5dea187d3b73"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:58.569864 30680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "b2e3251de3c044a98343877d8325a469"
format_stamp: "Formatted at 2026-08-06 16:19:58 on dist-test-slave-t3q3"
server_key: "45b3add53aa6a97ef3bbe31fa8e943de"
server_key_iv: "be56bdc1c38b2760ca1d5dea187d3b73"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:58.585880 30680 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.005s sys 0.001s
I20260806 16:19:58.620823 30700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:58.623394 30680 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.002s sys 0.000s
I20260806 16:19:58.623757 30680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "b2e3251de3c044a98343877d8325a469"
format_stamp: "Formatted at 2026-08-06 16:19:58 on dist-test-slave-t3q3"
server_key: "45b3add53aa6a97ef3bbe31fa8e943de"
server_key_iv: "be56bdc1c38b2760ca1d5dea187d3b73"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:58.624116 30680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:19:58.660697 30680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:19:58.668629 30680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:19:58.668954 30680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:19:58.670030 30680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:19:58.671492 30680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:19:58.671569 30680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:58.671639 30680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:19:58.671670 30680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:58.688943 30680 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:39143
I20260806 16:19:58.690758 30680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:19:58.700585 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 30680
I20260806 16:19:58.700788 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:19:58.701202 12246 external_mini_cluster.cc:1428] Setting key 6f9987ff108c8354d991c93582c369f4
I20260806 16:19:58.692655 30856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:39143 every 8 connection(s)
Aug 06 16:19:58 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033198, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:19:58.726948 30860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:58.700331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:47181 (local address 127.11.245.190:33055)
0806 16:19:58.707329 (+ 6998us) server_negotiation.cc:206] Beginning negotiation
0806 16:19:58.707335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:19:58.708101 (+ 766us) server_negotiation.cc:407] Connection header received
0806 16:19:58.709780 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:19:58.709784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:19:58.709859 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:19:58.709951 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:19:58.711961 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:58.713287 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:58.715268 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:58.715647 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:58.715736 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:58.721738 (+ 6002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:19:58.721743 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:19:58.721746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:19:58.721785 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:19:58.723865 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:58.724702 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:58.724707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:58.724709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:58.724815 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:58.725283 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:58.725287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:58.725289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:58.725596 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:19:58.725690 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 16:19:58.726560 (+ 870us) server_negotiation.cc:299] Negotiation successful
0806 16:19:58.726730 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6861,"thread_start_us":146,"threads_started":1}
I20260806 16:19:58.731683 30859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:58.695758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:33055 (local address 127.11.245.131:47181)
0806 16:19:58.707840 (+ 12082us) negotiation.cc:107] Waiting for socket to connect
0806 16:19:58.707916 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 16:19:58.709343 (+ 1427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:19:58.710390 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:19:58.710404 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:19:58.710883 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:19:58.711754 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:58.711776 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:58.713506 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:58.713510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:58.715089 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:58.715103 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:58.716042 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:58.716046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:58.716320 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:58.719261 (+ 2941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:19:58.719287 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:19:58.721473 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:19:58.724051 (+ 2578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:19:58.724082 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:19:58.724111 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:19:58.724563 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:19:58.724935 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:19:58.724938 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:19:58.724940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:19:58.725164 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:19:58.725769 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:19:58.725777 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:19:58.726215 (+ 438us) client_negotiation.cc:769] Sending connection context
0806 16:19:58.727586 (+ 1371us) client_negotiation.cc:240] Negotiation successful
0806 16:19:58.728107 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11736,"thread_start_us":239,"threads_started":1}
I20260806 16:19:58.733791 30857 heartbeater.cc:347] Connected to a master server at 127.11.245.190:33055
I20260806 16:19:58.734164 30857 heartbeater.cc:464] Registering TS with master...
I20260806 16:19:58.735035 30857 heartbeater.cc:511] Master 127.11.245.190:33055 requested a full tablet report, sending...
I20260806 16:19:58.736788 30193 ts_manager.cc:194] Registered new tserver with Master: b2e3251de3c044a98343877d8325a469 (127.11.245.131:39143)
I20260806 16:19:58.737726 30193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:47181
I20260806 16:19:58.739399 30481 heartbeater.cc:503] Master 127.11.245.190:33055 was elected leader, sending a full tablet report...
I20260806 16:19:58.743054 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:19:58.744001 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 30295
I20260806 16:19:58.769618 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 30488
I20260806 16:19:58.806466 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 30680
Aug 06 16:19:58 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033197, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
I20260806 16:19:58.836546 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 30151
I20260806 16:19:58.860850 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:33055
--webserver_interface=127.11.245.190
--webserver_port=37737
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:33055
--ranger_config_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.245.148:44529/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.245.148:44529/kms with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:19:59.031494 30863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:19:59.031857 30863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:19:59.031900 30863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:19:59.036197 30863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:19:59.036278 30863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:19:59.036296 30863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:19:59.036314 30863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:19:59.036330 30863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:19:59.043356 30863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.245.148:44529/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:33055
--ranger_config_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:33055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=37737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:19:59.045009 30863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:19:59.046424 30863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:19:59.057695 30876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:59.057701 30871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:59.057714 30872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:19:59.057922 30863 server_base.cc:1034] running on GCE node
I20260806 16:19:59.058864 30863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:19:59.060411 30863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:19:59.061622 30863 hybrid_clock.cc:648] HybridClock initialized: now 1786033199061603 us; error 36 us; skew 500 ppm
Aug 06 16:19:59 dist-test-slave-t3q3 krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033199, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:19:59.065130 30863 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:19:59.066256 30863 webserver.cc:460] Webserver started at http://127.11.245.190:37737/ using document root <none> and password file <none>
I20260806 16:19:59.066913 30863 fs_manager.cc:359] Metadata directory not provided
I20260806 16:19:59.066984 30863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 16:19:59 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033199, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
I20260806 16:19:59.122455 30863 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:19:59.126091 30886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:59.128178 30863 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:19:59.128376 30863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "96580ebc0f934b2db6991a8a0db698b9"
format_stamp: "Formatted at 2026-08-06 16:19:56 on dist-test-slave-t3q3"
server_key: "f4d7f6d8fc009b01177fac1bcc2e309e"
server_key_iv: "a1d08312316394f377304951df3f15fd"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:59.129339 30863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:19:59.154608 30863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:19:59.159264 30863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:19:59.159651 30863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:19:59.169216 30863 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:33055
I20260806 16:19:59.169232 30958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:33055 every 8 connection(s)
I20260806 16:19:59.170773 30863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:19:59.175678 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 30863
I20260806 16:19:59.176376 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.245.148:44529/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:43119
--local_ip_for_outbound_sockets=127.11.245.129
--tserver_master_addrs=127.11.245.190:33055
--webserver_port=40617
--webserver_interface=127.11.245.129
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.245.148:44529/kms with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:19:59.178218 30959 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:19:59.181049 30959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9: Bootstrap starting.
I20260806 16:19:59.202805 30959 log.cc:824] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9: Log is configured to *not* fsync() on all Append() calls
I20260806 16:19:59.211835 30959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:19:59.212606 30959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9: Bootstrap complete.
I20260806 16:19:59.218202 30959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96580ebc0f934b2db6991a8a0db698b9" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33055 } }
I20260806 16:19:59.218770 30959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96580ebc0f934b2db6991a8a0db698b9, State: Initialized, Role: FOLLOWER
I20260806 16:19:59.219496 30959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96580ebc0f934b2db6991a8a0db698b9" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33055 } }
I20260806 16:19:59.219707 30959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:19:59.219805 30959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:19:59.219947 30959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 1 FOLLOWER]: Advancing to term 2
I20260806 16:19:59.222649 30959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96580ebc0f934b2db6991a8a0db698b9" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33055 } }
I20260806 16:19:59.223105 30959 leader_election.cc:302] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96580ebc0f934b2db6991a8a0db698b9; no voters:
I20260806 16:19:59.223578 30959 leader_election.cc:288] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 16:19:59.223701 30964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 16:19:59.224148 30964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [term 2 LEADER]: Becoming Leader. State: Replica: 96580ebc0f934b2db6991a8a0db698b9, State: Running, Role: LEADER
I20260806 16:19:59.224524 30964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96580ebc0f934b2db6991a8a0db698b9" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33055 } }
I20260806 16:19:59.225282 30959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:19:59.226436 30965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "96580ebc0f934b2db6991a8a0db698b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96580ebc0f934b2db6991a8a0db698b9" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33055 } } }
I20260806 16:19:59.226584 30965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [sys.catalog]: This master's current role is: LEADER
I20260806 16:19:59.226500 30967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96580ebc0f934b2db6991a8a0db698b9. Latest consensus state: current_term: 2 leader_uuid: "96580ebc0f934b2db6991a8a0db698b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96580ebc0f934b2db6991a8a0db698b9" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33055 } } }
I20260806 16:19:59.226665 30967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9 [sys.catalog]: This master's current role is: LEADER
I20260806 16:19:59.228448 30959 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 16:19:59.228540 30973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:19:59.237604 30973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:19:59.238485 30973 catalog_manager.cc:1259] Loaded cluster ID: 242cf77fab1443a683065feb7ae13a1e
I20260806 16:19:59.238559 30973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:19:59.241412 30973 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:19:59.242224 30973 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9: Loaded TSK: 0
I20260806 16:19:59.243739 30973 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 16:19:59.367974 30962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:19:59.368520 30962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:19:59.368670 30962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:19:59.375779 30962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:19:59.375914 30962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:19:59.376024 30962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:19:59.386276 30962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.245.148:44529/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:43119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=40617
--tserver_master_addrs=127.11.245.190:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:19:59.388407 30962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:19:59.390616 30962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:19:59.407716 31009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:19:59.408038 30962 server_base.cc:1034] running on GCE node
W20260806 16:19:59.408294 31006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:59.408407 31005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:19:59.409343 30962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:19:59.410540 30962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:19:59.411748 30962 hybrid_clock.cc:648] HybridClock initialized: now 1786033199411728 us; error 50 us; skew 500 ppm
Aug 06 16:19:59 dist-test-slave-t3q3 krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033199, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:19:59.416689 30962 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:19:59.418475 30962 webserver.cc:460] Webserver started at http://127.11.245.129:40617/ using document root <none> and password file <none>
I20260806 16:19:59.419525 30962 fs_manager.cc:359] Metadata directory not provided
I20260806 16:19:59.419654 30962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 16:19:59 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033199, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
I20260806 16:19:59.450507 30962 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:19:59.454779 31019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:59.461711 30962 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.000s
I20260806 16:19:59.461979 30962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "84783f594c914ad8ab166b8a0ac6daf7"
format_stamp: "Formatted at 2026-08-06 16:19:57 on dist-test-slave-t3q3"
server_key: "ac592f36ac0098cdd57f7bf89bbfa302"
server_key_iv: "e5ff89b41ba27cbd56f915a4c006c477"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:59.462932 30962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:19:59.494725 30962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:19:59.499946 30962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:19:59.500295 30962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:19:59.501356 30962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:19:59.502861 30962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:19:59.502938 30962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:59.503029 30962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:19:59.503065 30962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:59.521623 30962 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:43119
I20260806 16:19:59.521654 31175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:43119 every 8 connection(s)
I20260806 16:19:59.523173 30962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:19:59.529206 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 30962
I20260806 16:19:59.529784 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.245.148:44529/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:36601
--local_ip_for_outbound_sockets=127.11.245.130
--tserver_master_addrs=127.11.245.190:33055
--webserver_port=34917
--webserver_interface=127.11.245.130
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.245.148:44529/kms with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:19:59 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033199, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:19:59.549082 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:59.528800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:35085 (local address 127.11.245.190:33055)
0806 16:19:59.530737 (+ 1937us) server_negotiation.cc:206] Beginning negotiation
0806 16:19:59.530756 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:19:59.530803 (+ 47us) server_negotiation.cc:407] Connection header received
0806 16:19:59.531098 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:19:59.531110 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:19:59.531485 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:19:59.531884 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:19:59.533634 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:59.535536 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:59.537152 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:59.537741 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:59.537925 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:59.542018 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:19:59.542027 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:19:59.542050 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:19:59.542098 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:19:59.544639 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:59.545377 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:59.545384 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:59.545389 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:59.545537 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:59.545957 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:59.545961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:59.545963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:59.546491 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:19:59.546678 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 16:19:59.547375 (+ 697us) server_negotiation.cc:299] Negotiation successful
0806 16:19:59.547707 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1751,"thread_start_us":131,"threads_started":1}
I20260806 16:19:59.549098 31178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:59.527382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:33055 (local address 127.11.245.129:35085)
0806 16:19:59.528413 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0806 16:19:59.528488 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 16:19:59.530209 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:19:59.531968 (+ 1759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:19:59.531977 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:19:59.532467 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:19:59.533280 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:59.533307 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:59.535763 (+ 2456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:59.535767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:59.536986 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:59.537006 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:59.538213 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:59.538217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:59.538424 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:59.539321 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:19:59.539348 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:19:59.541671 (+ 2323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:19:59.544828 (+ 3157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:19:59.544836 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:19:59.544854 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:19:59.545196 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:19:59.545654 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:19:59.545657 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:19:59.545660 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:19:59.545840 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:19:59.546641 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:19:59.546650 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:19:59.546989 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 16:19:59.547229 (+ 240us) client_negotiation.cc:240] Negotiation successful
0806 16:19:59.547707 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":208,"threads_started":1}
I20260806 16:19:59.552942 31176 heartbeater.cc:347] Connected to a master server at 127.11.245.190:33055
I20260806 16:19:59.553339 31176 heartbeater.cc:464] Registering TS with master...
I20260806 16:19:59.554111 31176 heartbeater.cc:511] Master 127.11.245.190:33055 requested a full tablet report, sending...
W20260806 16:19:59.557127 31176 heartbeater.cc:644] Failed to heartbeat to 127.11.245.190:33055 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 16:19:59.557302 31176 heartbeater.cc:464] Registering TS with master...
I20260806 16:19:59.557561 31176 heartbeater.cc:511] Master 127.11.245.190:33055 requested a full tablet report, sending...
I20260806 16:19:59.558676 31176 heartbeater.cc:464] Registering TS with master...
I20260806 16:19:59.558887 31176 heartbeater.cc:511] Master 127.11.245.190:33055 requested a full tablet report, sending...
W20260806 16:19:59.559861 31176 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:19:59.733508 31183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:19:59.733935 31183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:19:59.734014 31183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:19:59.741200 31183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:19:59.741357 31183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:19:59.741612 31183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:19:59.749524 31183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.245.148:44529/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:36601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=34917
--tserver_master_addrs=127.11.245.190:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:19:59.751232 31183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:19:59.752986 31183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:19:59.769498 31195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:19:59.772691 31183 server_base.cc:1034] running on GCE node
W20260806 16:19:59.772830 31190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:19:59.772715 31191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:19:59.773775 31183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:19:59.774941 31183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:19:59.778420 31183 hybrid_clock.cc:648] HybridClock initialized: now 1786033199778325 us; error 80 us; skew 500 ppm
Aug 06 16:19:59 dist-test-slave-t3q3 krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033199, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:19:59.789004 31183 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:19:59.790549 31183 webserver.cc:460] Webserver started at http://127.11.245.130:34917/ using document root <none> and password file <none>
I20260806 16:19:59.791368 31183 fs_manager.cc:359] Metadata directory not provided
I20260806 16:19:59.791501 31183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 16:19:59 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033199, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
I20260806 16:19:59.819144 31183 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:19:59.832090 31208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:59.834765 31183 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.002s sys 0.000s
I20260806 16:19:59.834985 31183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "cf996ba5c1d0485c9b59dd83e01baab4"
format_stamp: "Formatted at 2026-08-06 16:19:58 on dist-test-slave-t3q3"
server_key: "74d82c08206358f10d4dc01548035d38"
server_key_iv: "9b4cbfa59427abfb61d338b15fa6f106"
server_key_version: "kuduclusterkey@0"
I20260806 16:19:59.835831 31183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:19:59.860383 31183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:19:59.866691 31183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:19:59.867023 31183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:19:59.868280 31183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:19:59.869971 31183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:19:59.870078 31183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:59.870185 31183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:19:59.870237 31183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:19:59.884524 31183 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:36601
I20260806 16:19:59.884670 31376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:36601 every 8 connection(s)
I20260806 16:19:59.886399 31183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:19:59.896274 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 31183
I20260806 16:19:59.896972 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.245.148:44529/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:39143
--local_ip_for_outbound_sockets=127.11.245.131
--tserver_master_addrs=127.11.245.190:33055
--webserver_port=46645
--webserver_interface=127.11.245.131
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.245.148:44529/kms with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:19:59 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033199, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:19:59.916507 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:59.889438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:47221 (local address 127.11.245.190:33055)
0806 16:19:59.889671 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 16:19:59.889677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:19:59.892148 (+ 2471us) server_negotiation.cc:407] Connection header received
0806 16:19:59.893969 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:19:59.893974 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:19:59.894065 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:19:59.894139 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:19:59.900808 (+ 6669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:59.901949 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:59.904491 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:19:59.904926 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:19:59.905059 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:59.910056 (+ 4997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:19:59.910065 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:19:59.910072 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:19:59.910133 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:19:59.912700 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:59.913840 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:59.913846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:59.913848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:59.914002 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:19:59.914577 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:19:59.914582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:19:59.914584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:19:59.914948 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:19:59.915065 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 16:19:59.916087 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0806 16:19:59.916287 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 16:19:59.918896 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:19:59.890118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:33055 (local address 127.11.245.130:47221)
0806 16:19:59.891834 (+ 1716us) negotiation.cc:107] Waiting for socket to connect
0806 16:19:59.891929 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0806 16:19:59.893586 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:19:59.894742 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:19:59.894755 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:19:59.895281 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:19:59.900384 (+ 5103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:59.900432 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:59.902194 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:59.902198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:59.904215 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:19:59.904240 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:19:59.905405 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:19:59.905409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:19:59.905693 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:19:59.906748 (+ 1055us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:19:59.906780 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:19:59.909342 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:19:59.912948 (+ 3606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:19:59.912963 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:19:59.913017 (+ 54us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:19:59.913640 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:19:59.914152 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:19:59.914156 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:19:59.914158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:19:59.914431 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:19:59.915154 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:19:59.915165 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:19:59.915668 (+ 503us) client_negotiation.cc:769] Sending connection context
0806 16:19:59.916867 (+ 1199us) client_negotiation.cc:240] Negotiation successful
0806 16:19:59.917407 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1399,"thread_start_us":200,"threads_started":1}
I20260806 16:19:59.920925 31377 heartbeater.cc:347] Connected to a master server at 127.11.245.190:33055
I20260806 16:19:59.921298 31377 heartbeater.cc:464] Registering TS with master...
I20260806 16:19:59.922174 31377 heartbeater.cc:511] Master 127.11.245.190:33055 requested a full tablet report, sending...
W20260806 16:19:59.924805 31377 heartbeater.cc:644] Failed to heartbeat to 127.11.245.190:33055 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 16:19:59.925010 31377 heartbeater.cc:464] Registering TS with master...
I20260806 16:19:59.925300 31377 heartbeater.cc:511] Master 127.11.245.190:33055 requested a full tablet report, sending...
I20260806 16:19:59.926649 31377 heartbeater.cc:464] Registering TS with master...
I20260806 16:19:59.926906 31377 heartbeater.cc:511] Master 127.11.245.190:33055 requested a full tablet report, sending...
W20260806 16:19:59.927974 31377 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:20:00.122819 31383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:00.123395 31383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:00.123485 31383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:00.130532 31383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:00.130654 31383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:00.130779 31383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:20:00.141016 31383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.245.148:44529/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:39143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=46645
--tserver_master_addrs=127.11.245.190:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:00.143236 31383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:00.145251 31383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:00.162001 31394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:00.162375 31383 server_base.cc:1034] running on GCE node
W20260806 16:20:00.162456 31393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:00.162587 31397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:00.163134 31383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:00.164271 31383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:00.165511 31383 hybrid_clock.cc:648] HybridClock initialized: now 1786033200165476 us; error 59 us; skew 500 ppm
Aug 06 16:20:00 dist-test-slave-t3q3 krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033200, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:00.169605 31383 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:20:00.171342 31383 webserver.cc:460] Webserver started at http://127.11.245.131:46645/ using document root <none> and password file <none>
I20260806 16:20:00.172705 31383 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:00.172847 31383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 16:20:00 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033200, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
I20260806 16:20:00.202042 31383 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.001s
I20260806 16:20:00.206236 31415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:00.208432 31383 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
I20260806 16:20:00.208635 31383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "b2e3251de3c044a98343877d8325a469"
format_stamp: "Formatted at 2026-08-06 16:19:58 on dist-test-slave-t3q3"
server_key: "45b3add53aa6a97ef3bbe31fa8e943de"
server_key_iv: "be56bdc1c38b2760ca1d5dea187d3b73"
server_key_version: "kuduclusterkey@0"
I20260806 16:20:00.209683 31383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:00.236121 31383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:00.241933 31383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:00.242259 31383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:00.243353 31383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:20:00.244922 31383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:20:00.245000 31383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:00.245067 31383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:20:00.245090 31383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:00.268784 31383 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:39143
I20260806 16:20:00.270658 31383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:20:00.272008 31576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:39143 every 8 connection(s)
I20260806 16:20:00.272511 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 31383
I20260806 16:20:00.290257 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:00.274014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44626 (local address 127.11.245.190:33055)
0806 16:20:00.274440 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:00.274446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:00.274468 (+ 22us) server_negotiation.cc:407] Connection header received
0806 16:20:00.274609 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:00.274613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:00.274681 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:00.274827 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:00.275919 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:00.276847 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:00.278230 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:00.278627 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:00.278757 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:00.282833 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:00.282838 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:00.282841 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:00.282881 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:00.285098 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:00.288168 (+ 3070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:00.288173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:00.288175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:00.288321 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:00.289067 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:00.289072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:00.289074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:00.289363 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:00.289651 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:00.289866 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 16:20:00.290036 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324}
Aug 06 16:20:00 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033200, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:00.297505 31582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:00.274129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:59221 (local address 127.11.245.190:33055)
0806 16:20:00.274440 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:00.274453 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:00.276433 (+ 1980us) server_negotiation.cc:407] Connection header received
0806 16:20:00.282319 (+ 5886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:00.282359 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:00.282470 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:00.282563 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:00.285019 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:00.285870 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:00.287287 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:00.287560 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:00.287687 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:00.292841 (+ 5154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:00.292847 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:00.292850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:00.292881 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:00.294632 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:00.295357 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:00.295361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:00.295362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:00.295459 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:00.295899 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:00.295902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:00.295904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:00.296205 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:00.296321 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:00.297100 (+ 779us) server_negotiation.cc:299] Negotiation successful
0806 16:20:00.297270 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":151,"threads_started":1}
I20260806 16:20:00.299044 31584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:00.274314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:33055 (local address 127.11.245.131:59221)
0806 16:20:00.276238 (+ 1924us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:00.276313 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:00.281710 (+ 5397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:00.283274 (+ 1564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:00.283304 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:00.283815 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:00.284808 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:00.284833 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:00.286037 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:00.286042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:00.287123 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:00.287137 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:00.287770 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:00.287775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:00.287994 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:00.289174 (+ 1180us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:00.289203 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:00.292586 (+ 3383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:00.294800 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:00.294810 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:00.294846 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:00.295224 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:00.295562 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:00.295566 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:00.295568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:00.295784 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:00.296322 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:00.296336 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:00.296780 (+ 444us) client_negotiation.cc:769] Sending connection context
0806 16:20:00.297049 (+ 269us) client_negotiation.cc:240] Negotiation successful
0806 16:20:00.297519 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1587,"thread_start_us":209,"threads_started":1}
I20260806 16:20:00.300904 31578 heartbeater.cc:347] Connected to a master server at 127.11.245.190:33055
I20260806 16:20:00.301301 31578 heartbeater.cc:464] Registering TS with master...
I20260806 16:20:00.302075 31578 heartbeater.cc:511] Master 127.11.245.190:33055 requested a full tablet report, sending...
I20260806 16:20:00.304117 30910 ts_manager.cc:194] Registered new tserver with Master: b2e3251de3c044a98343877d8325a469 (127.11.245.131:39143)
I20260806 16:20:00.305732 30910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:59221
I20260806 16:20:00.560262 31176 heartbeater.cc:464] Registering TS with master...
I20260806 16:20:00.560748 31176 heartbeater.cc:511] Master 127.11.245.190:33055 requested a full tablet report, sending...
I20260806 16:20:00.561949 30910 ts_manager.cc:194] Registered new tserver with Master: 84783f594c914ad8ab166b8a0ac6daf7 (127.11.245.129:43119)
I20260806 16:20:00.562947 30910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:35085
Aug 06 16:20:00 dist-test-slave-t3q3 krb5kdc[29532](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786033200, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:00.928372 31377 heartbeater.cc:464] Registering TS with master...
I20260806 16:20:00.928860 31377 heartbeater.cc:511] Master 127.11.245.190:33055 requested a full tablet report, sending...
I20260806 16:20:00.934367 30910 ts_manager.cc:194] Registered new tserver with Master: cf996ba5c1d0485c9b59dd83e01baab4 (127.11.245.130:36601)
I20260806 16:20:00.935330 30910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:47221
I20260806 16:20:00.939380 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:20:00.954420 31598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:00.941823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44628 (local address 127.11.245.190:33055)
0806 16:20:00.942910 (+ 1087us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:00.942916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:00.942933 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:20:00.942990 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:00.942994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:00.943057 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:00.944649 (+ 1592us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:00.944797 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:00.945694 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:00.947341 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:00.947748 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:00.948563 (+ 815us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:00.950190 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:00.950194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:00.950196 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:00.950258 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:00.952186 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:00.952802 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:00.952806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:00.952808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:00.952925 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:00.953266 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:00.953270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:00.953272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:00.953546 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:00.953715 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:00.954023 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 16:20:00.954187 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":987,"thread_start_us":106,"threads_started":1}
I20260806 16:20:00.959683 30910 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:20:00.965027 30910 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:20:00.997345 31610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:00.988464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:36601 (local address 127.0.0.1:48970)
0806 16:20:00.990611 (+ 2147us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:00.990628 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:00.990839 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:00.991820 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:00.991828 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:00.991859 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:00.992085 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:00.992096 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:00.993660 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:00.993663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:00.995125 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:00.995137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:00.996633 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:00.996636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:00.996777 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:00.996830 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 16:20:00.996966 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 16:20:00.997141 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2064,"thread_start_us":124,"threads_started":1}
I20260806 16:20:00.998199 31609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:00.989020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48970 (local address 127.11.245.130:36601)
0806 16:20:00.989442 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:00.989452 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:00.990741 (+ 1289us) server_negotiation.cc:407] Connection header received
0806 16:20:00.991031 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:00.991038 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:00.991359 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:00.991541 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:00.992238 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:00.993507 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:00.995275 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:00.996207 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:00.996319 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:00.996521 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:00.997861 (+ 1340us) server_negotiation.cc:299] Negotiation successful
0806 16:20:00.998030 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":203,"threads_started":1}
I20260806 16:20:00.998489 31604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:00.985954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:43119 (local address 127.0.0.1:46628)
0806 16:20:00.988511 (+ 2557us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:00.988597 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:00.988789 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:00.992670 (+ 3881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:00.992675 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:00.992695 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:00.992805 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:00.992811 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:00.994465 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:00.994469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:00.995563 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:00.995574 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:00.998074 (+ 2500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:00.998078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:00.998167 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:00.998195 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 16:20:00.998255 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 16:20:00.998354 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2281,"thread_start_us":182,"threads_started":1}
I20260806 16:20:01.003535 31283 tablet_service.cc:1467] Processing CreateTablet for tablet 707d037284d2466ebb0092ca0145adb5 (DEFAULT_TABLE table=test-table [id=0c8bbe14b9914cbabcac266993f2aa10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:20:01.005405 31283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707d037284d2466ebb0092ca0145adb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:01.006580 31604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:00.990316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:39143 (local address 127.0.0.1:50382)
0806 16:20:00.998766 (+ 8450us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:00.998783 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:00.998872 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:01.001870 (+ 2998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:01.001874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:01.001895 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:01.002021 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:01.002028 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.003761 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.003765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:01.004887 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:01.004903 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.006130 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.006134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:01.006228 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:01.006266 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 16:20:01.006322 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 16:20:01.006409 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8385,"thread_start_us":114,"threads_started":1}
I20260806 16:20:01.008659 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:00.991107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50382 (local address 127.11.245.131:39143)
0806 16:20:00.991820 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:00.991829 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:01.000225 (+ 8396us) server_negotiation.cc:407] Connection header received
0806 16:20:01.000357 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:01.000364 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:01.000638 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:01.002235 (+ 1597us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:01.002450 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.003607 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.005097 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.005957 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.008113 (+ 2156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:01.008305 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:01.008369 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 16:20:01.008490 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":169,"threads_started":1}
I20260806 16:20:01.010535 31606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:00.987293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46628 (local address 127.11.245.129:43119)
0806 16:20:00.992102 (+ 4809us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:00.992109 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:00.992140 (+ 31us) server_negotiation.cc:407] Connection header received
0806 16:20:00.992240 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:00.992245 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:00.992475 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:00.992977 (+ 502us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:00.993176 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:00.994322 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:00.997039 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:00.997905 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.009929 (+ 12024us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:01.010153 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:01.010225 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 16:20:01.010360 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4640,"thread_start_us":205,"threads_started":1}
I20260806 16:20:01.013043 31487 tablet_service.cc:1467] Processing CreateTablet for tablet 707d037284d2466ebb0092ca0145adb5 (DEFAULT_TABLE table=test-table [id=0c8bbe14b9914cbabcac266993f2aa10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:20:01.013983 31087 tablet_service.cc:1467] Processing CreateTablet for tablet 707d037284d2466ebb0092ca0145adb5 (DEFAULT_TABLE table=test-table [id=0c8bbe14b9914cbabcac266993f2aa10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:20:01.014750 31487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707d037284d2466ebb0092ca0145adb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:01.015164 31087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707d037284d2466ebb0092ca0145adb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:01.032356 31616 tablet_bootstrap.cc:492] T 707d037284d2466ebb0092ca0145adb5 P 84783f594c914ad8ab166b8a0ac6daf7: Bootstrap starting.
I20260806 16:20:01.038187 31616 tablet_bootstrap.cc:654] T 707d037284d2466ebb0092ca0145adb5 P 84783f594c914ad8ab166b8a0ac6daf7: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:01.039412 31616 log.cc:824] T 707d037284d2466ebb0092ca0145adb5 P 84783f594c914ad8ab166b8a0ac6daf7: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:01.040586 31617 tablet_bootstrap.cc:492] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4: Bootstrap starting.
I20260806 16:20:01.043839 31617 tablet_bootstrap.cc:654] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:01.044605 31615 tablet_bootstrap.cc:492] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469: Bootstrap starting.
I20260806 16:20:01.045010 31617 log.cc:824] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:01.047310 31615 tablet_bootstrap.cc:654] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:01.048368 31615 log.cc:824] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:01.048856 31617 tablet_bootstrap.cc:492] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4: No bootstrap required, opened a new log
I20260806 16:20:01.049121 31617 ts_tablet_manager.cc:1397] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 16:20:01.050887 31616 tablet_bootstrap.cc:492] T 707d037284d2466ebb0092ca0145adb5 P 84783f594c914ad8ab166b8a0ac6daf7: No bootstrap required, opened a new log
I20260806 16:20:01.052629 31616 ts_tablet_manager.cc:1397] T 707d037284d2466ebb0092ca0145adb5 P 84783f594c914ad8ab166b8a0ac6daf7: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.000s
I20260806 16:20:01.053895 31617 raft_consensus.cc:348] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84783f594c914ad8ab166b8a0ac6daf7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43119 } } peers { permanent_uuid: "cf996ba5c1d0485c9b59dd83e01baab4" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 36601 } } peers { permanent_uuid: "b2e3251de3c044a98343877d8325a469" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39143 } }
I20260806 16:20:01.054278 31617 raft_consensus.cc:374] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:01.054333 31617 raft_consensus.cc:729] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf996ba5c1d0485c9b59dd83e01baab4, State: Initialized, Role: FOLLOWER
I20260806 16:20:01.054944 31617 consensus_queue.cc:260] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84783f594c914ad8ab166b8a0ac6daf7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43119 } } peers { permanent_uuid: "cf996ba5c1d0485c9b59dd83e01baab4" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 36601 } } peers { permanent_uuid: "b2e3251de3c044a98343877d8325a469" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39143 } }
I20260806 16:20:01.056357 31617 ts_tablet_manager.cc:1428] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 16:20:01.056915 31377 heartbeater.cc:503] Master 127.11.245.190:33055 was elected leader, sending a full tablet report...
I20260806 16:20:01.059442 31615 tablet_bootstrap.cc:492] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469: No bootstrap required, opened a new log
I20260806 16:20:01.059706 31615 ts_tablet_manager.cc:1397] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
I20260806 16:20:01.064818 31615 raft_consensus.cc:348] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84783f594c914ad8ab166b8a0ac6daf7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43119 } } peers { permanent_uuid: "cf996ba5c1d0485c9b59dd83e01baab4" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 36601 } } peers { permanent_uuid: "b2e3251de3c044a98343877d8325a469" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39143 } }
I20260806 16:20:01.065209 31615 raft_consensus.cc:374] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:01.065263 31615 raft_consensus.cc:729] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e3251de3c044a98343877d8325a469, State: Initialized, Role: FOLLOWER
I20260806 16:20:01.065918 31615 consensus_queue.cc:260] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84783f594c914ad8ab166b8a0ac6daf7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43119 } } peers { permanent_uuid: "cf996ba5c1d0485c9b59dd83e01baab4" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 36601 } } peers { permanent_uuid: "b2e3251de3c044a98343877d8325a469" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39143 } }
I20260806 16:20:01.067332 31615 ts_tablet_manager.cc:1428] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260806 16:20:01.067818 31578 heartbeater.cc:503] Master 127.11.245.190:33055 was elected leader, sending a full tablet report...
I20260806 16:20:01.070266 31616 raft_consensus.cc:348] T 707d037284d2466ebb0092ca0145adb5 P 84783f594c914ad8ab166b8a0ac6daf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84783f594c914ad8ab166b8a0ac6daf7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43119 } } peers { permanent_uuid: "cf996ba5c1d0485c9b59dd83e01baab4" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 36601 } } peers { permanent_uuid: "b2e3251de3c044a98343877d8325a469" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39143 } }
I20260806 16:20:01.070755 31616 raft_consensus.cc:374] T 707d037284d2466ebb0092ca0145adb5 P 84783f594c914ad8ab166b8a0ac6daf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:01.070894 31616 raft_consensus.cc:729] T 707d037284d2466ebb0092ca0145adb5 P 84783f594c914ad8ab166b8a0ac6daf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84783f594c914ad8ab166b8a0ac6daf7, State: Initialized, Role: FOLLOWER
I20260806 16:20:01.071606 31616 consensus_queue.cc:260] T 707d037284d2466ebb0092ca0145adb5 P 84783f594c914ad8ab166b8a0ac6daf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84783f594c914ad8ab166b8a0ac6daf7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43119 } } peers { permanent_uuid: "cf996ba5c1d0485c9b59dd83e01baab4" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 36601 } } peers { permanent_uuid: "b2e3251de3c044a98343877d8325a469" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39143 } }
I20260806 16:20:01.072729 31176 heartbeater.cc:503] Master 127.11.245.190:33055 was elected leader, sending a full tablet report...
I20260806 16:20:01.073246 31616 ts_tablet_manager.cc:1428] T 707d037284d2466ebb0092ca0145adb5 P 84783f594c914ad8ab166b8a0ac6daf7: Time spent starting tablet: real 0.020s user 0.007s sys 0.001s
Aug 06 16:20:01 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033200, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
W20260806 16:20:01.139194 31378 tablet.cc:2369] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:20:01.274394 31579 tablet.cc:2369] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:20:01.276571 31177 tablet.cc:2369] T 707d037284d2466ebb0092ca0145adb5 P 84783f594c914ad8ab166b8a0ac6daf7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:20:01.376644 31622 raft_consensus.cc:482] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:20:01.377024 31622 raft_consensus.cc:504] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84783f594c914ad8ab166b8a0ac6daf7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43119 } } peers { permanent_uuid: "cf996ba5c1d0485c9b59dd83e01baab4" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 36601 } } peers { permanent_uuid: "b2e3251de3c044a98343877d8325a469" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39143 } }
I20260806 16:20:01.378958 31622 leader_election.cc:288] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84783f594c914ad8ab166b8a0ac6daf7 (127.11.245.129:43119), cf996ba5c1d0485c9b59dd83e01baab4 (127.11.245.130:36601)
I20260806 16:20:01.394104 31606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:01.380693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:59389 (local address 127.11.245.129:43119)
0806 16:20:01.380857 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:01.380862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:01.388588 (+ 7726us) server_negotiation.cc:407] Connection header received
0806 16:20:01.388912 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:01.388916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:01.389173 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:01.389258 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:01.389864 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.390653 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.392414 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.393111 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.393258 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:01.393339 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:01.393780 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 16:20:01.393913 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 16:20:01.394742 31625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:01.380463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:43119 (local address 127.11.245.131:59389)
0806 16:20:01.388310 (+ 7847us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:01.388333 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:01.388725 (+ 392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:01.389506 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:01.389513 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:01.389541 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:01.389711 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:01.389718 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.390839 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.390843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:01.392215 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:01.392229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.393472 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.393475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:01.393597 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:01.393632 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 16:20:01.394437 (+ 805us) client_negotiation.cc:240] Negotiation successful
0806 16:20:01.394535 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7733,"thread_start_us":114,"threads_started":1}
I20260806 16:20:01.396644 31114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707d037284d2466ebb0092ca0145adb5" candidate_uuid: "b2e3251de3c044a98343877d8325a469" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84783f594c914ad8ab166b8a0ac6daf7" is_pre_election: true
I20260806 16:20:01.397315 31114 raft_consensus.cc:2454] T 707d037284d2466ebb0092ca0145adb5 P 84783f594c914ad8ab166b8a0ac6daf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e3251de3c044a98343877d8325a469 in term 0.
I20260806 16:20:01.398088 31420 leader_election.cc:302] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84783f594c914ad8ab166b8a0ac6daf7, b2e3251de3c044a98343877d8325a469; no voters:
I20260806 16:20:01.398548 31622 raft_consensus.cc:2781] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:20:01.398650 31622 raft_consensus.cc:482] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:20:01.398696 31622 raft_consensus.cc:3037] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:01.400513 31622 raft_consensus.cc:504] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84783f594c914ad8ab166b8a0ac6daf7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43119 } } peers { permanent_uuid: "cf996ba5c1d0485c9b59dd83e01baab4" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 36601 } } peers { permanent_uuid: "b2e3251de3c044a98343877d8325a469" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39143 } }
I20260806 16:20:01.401067 31622 leader_election.cc:288] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [CANDIDATE]: Term 1 election: Requested vote from peers 84783f594c914ad8ab166b8a0ac6daf7 (127.11.245.129:43119), cf996ba5c1d0485c9b59dd83e01baab4 (127.11.245.130:36601)
I20260806 16:20:01.401886 31114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707d037284d2466ebb0092ca0145adb5" candidate_uuid: "b2e3251de3c044a98343877d8325a469" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84783f594c914ad8ab166b8a0ac6daf7"
I20260806 16:20:01.402117 31114 raft_consensus.cc:3037] T 707d037284d2466ebb0092ca0145adb5 P 84783f594c914ad8ab166b8a0ac6daf7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:01.403878 31114 raft_consensus.cc:2454] T 707d037284d2466ebb0092ca0145adb5 P 84783f594c914ad8ab166b8a0ac6daf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e3251de3c044a98343877d8325a469 in term 1.
I20260806 16:20:01.404524 31621 raft_consensus.cc:482] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:20:01.404891 31621 raft_consensus.cc:504] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84783f594c914ad8ab166b8a0ac6daf7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43119 } } peers { permanent_uuid: "cf996ba5c1d0485c9b59dd83e01baab4" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 36601 } } peers { permanent_uuid: "b2e3251de3c044a98343877d8325a469" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39143 } }
I20260806 16:20:01.406873 31621 leader_election.cc:288] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84783f594c914ad8ab166b8a0ac6daf7 (127.11.245.129:43119), b2e3251de3c044a98343877d8325a469 (127.11.245.131:39143)
I20260806 16:20:01.409534 31420 leader_election.cc:302] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84783f594c914ad8ab166b8a0ac6daf7, b2e3251de3c044a98343877d8325a469; no voters:
I20260806 16:20:01.409849 31622 raft_consensus.cc:2781] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:20:01.410176 31622 raft_consensus.cc:686] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [term 1 LEADER]: Becoming Leader. State: Replica: b2e3251de3c044a98343877d8325a469, State: Running, Role: LEADER
I20260806 16:20:01.410717 31622 consensus_queue.cc:237] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84783f594c914ad8ab166b8a0ac6daf7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43119 } } peers { permanent_uuid: "cf996ba5c1d0485c9b59dd83e01baab4" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 36601 } } peers { permanent_uuid: "b2e3251de3c044a98343877d8325a469" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39143 } }
I20260806 16:20:01.416847 30906 catalog_manager.cc:5495] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2e3251de3c044a98343877d8325a469 (127.11.245.131). New cstate: current_term: 1 leader_uuid: "b2e3251de3c044a98343877d8325a469" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84783f594c914ad8ab166b8a0ac6daf7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf996ba5c1d0485c9b59dd83e01baab4" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 36601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e3251de3c044a98343877d8325a469" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39143 } health_report { overall_health: HEALTHY } } }
I20260806 16:20:01.422133 31606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:01.408564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:33545 (local address 127.11.245.129:43119)
0806 16:20:01.408716 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:01.408721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:01.416560 (+ 7839us) server_negotiation.cc:407] Connection header received
0806 16:20:01.416871 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:01.416876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:01.416951 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:01.417021 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:01.417632 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.418469 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.420336 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.420975 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.421106 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:01.421181 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:01.421774 (+ 593us) server_negotiation.cc:299] Negotiation successful
0806 16:20:01.421932 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 16:20:01.422641 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:01.407366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:43119 (local address 127.11.245.130:33545)
0806 16:20:01.416390 (+ 9024us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:01.416415 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:01.416691 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:01.417215 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:01.417218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:01.417240 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:01.417464 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:01.417473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.418711 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.418715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:01.420163 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:01.420176 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.421472 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.421475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:01.421584 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:01.421616 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 16:20:01.422356 (+ 740us) client_negotiation.cc:240] Negotiation successful
0806 16:20:01.422465 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8888,"thread_start_us":189,"threads_started":1}
I20260806 16:20:01.426573 31114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707d037284d2466ebb0092ca0145adb5" candidate_uuid: "cf996ba5c1d0485c9b59dd83e01baab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84783f594c914ad8ab166b8a0ac6daf7" is_pre_election: true
I20260806 16:20:01.426888 31114 raft_consensus.cc:2379] T 707d037284d2466ebb0092ca0145adb5 P 84783f594c914ad8ab166b8a0ac6daf7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf996ba5c1d0485c9b59dd83e01baab4 in current term 1: Already voted for candidate b2e3251de3c044a98343877d8325a469 in this term.
I20260806 16:20:01.430626 31609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:01.380166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:36907 (local address 127.11.245.130:36601)
0806 16:20:01.380350 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:01.380357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:01.385754 (+ 5397us) server_negotiation.cc:407] Connection header received
0806 16:20:01.386068 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:01.386073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:01.386138 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:01.386192 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:01.386658 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.387498 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.394822 (+ 7324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.395452 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.395574 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:01.395632 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:01.430296 (+ 34664us) server_negotiation.cc:299] Negotiation successful
0806 16:20:01.430444 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 16:20:01.431303 31624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:01.379506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:36601 (local address 127.11.245.131:36907)
0806 16:20:01.385588 (+ 6082us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:01.385612 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:01.385889 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:01.386371 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:01.386374 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:01.386392 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:01.386527 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:01.386535 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.387681 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.387684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:01.394609 (+ 6925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:01.394620 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.429335 (+ 34715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.429340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:01.430082 (+ 742us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:01.430127 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 16:20:01.430940 (+ 813us) client_negotiation.cc:240] Negotiation successful
0806 16:20:01.431060 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5955,"thread_start_us":160,"threads_started":1}
I20260806 16:20:01.432314 31310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707d037284d2466ebb0092ca0145adb5" candidate_uuid: "b2e3251de3c044a98343877d8325a469" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf996ba5c1d0485c9b59dd83e01baab4" is_pre_election: true
I20260806 16:20:01.432703 31310 raft_consensus.cc:2454] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2e3251de3c044a98343877d8325a469 in term 0.
I20260806 16:20:01.433084 31309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707d037284d2466ebb0092ca0145adb5" candidate_uuid: "b2e3251de3c044a98343877d8325a469" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf996ba5c1d0485c9b59dd83e01baab4"
I20260806 16:20:01.433199 31309 raft_consensus.cc:3037] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:01.434183 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:01.408850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:33665 (local address 127.11.245.131:39143)
0806 16:20:01.409037 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:01.409043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:01.422959 (+ 13916us) server_negotiation.cc:407] Connection header received
0806 16:20:01.423214 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:01.423218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:01.423283 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:01.423337 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:01.423726 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.424446 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.432530 (+ 8084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.433233 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.433371 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:01.433441 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:01.433956 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 16:20:01.434053 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 16:20:01.434815 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:01.407966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:39143 (local address 127.11.245.130:33665)
0806 16:20:01.422865 (+ 14899us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:01.422886 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:01.423069 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:01.423496 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:01.423498 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:01.423517 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:01.423603 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:01.423608 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.424605 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.424608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:01.425708 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:01.425721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.433621 (+ 7900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.433625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:01.433790 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:01.433820 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 16:20:01.434470 (+ 650us) client_negotiation.cc:240] Negotiation successful
0806 16:20:01.434640 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14825,"thread_start_us":360,"threads_started":1}
I20260806 16:20:01.434942 31309 raft_consensus.cc:2454] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2e3251de3c044a98343877d8325a469 in term 1.
I20260806 16:20:01.435948 31509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707d037284d2466ebb0092ca0145adb5" candidate_uuid: "cf996ba5c1d0485c9b59dd83e01baab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e3251de3c044a98343877d8325a469" is_pre_election: true
I20260806 16:20:01.436882 31210 leader_election.cc:302] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf996ba5c1d0485c9b59dd83e01baab4; no voters: 84783f594c914ad8ab166b8a0ac6daf7, b2e3251de3c044a98343877d8325a469
I20260806 16:20:01.437517 31621 raft_consensus.cc:2726] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 16:20:01.470532 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:01.462979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50392 (local address 127.11.245.131:39143)
0806 16:20:01.463183 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:01.463189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:01.463957 (+ 768us) server_negotiation.cc:407] Connection header received
0806 16:20:01.464347 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:01.464352 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:01.464450 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:01.464559 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:20:01.465050 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.465922 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.467271 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:01.467640 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:01.467734 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:01.468453 (+ 719us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:20:01.469542 (+ 1089us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:20:01.469668 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:01.470192 (+ 524us) server_negotiation.cc:299] Negotiation successful
0806 16:20:01.470300 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 16:20:01.482489 31309 raft_consensus.cc:1261] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4 [term 1 FOLLOWER]: Refusing update from remote peer b2e3251de3c044a98343877d8325a469: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:20:01.483588 31622 consensus_queue.cc:1035] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf996ba5c1d0485c9b59dd83e01baab4" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 36601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:20:01.495424 31114 raft_consensus.cc:1261] T 707d037284d2466ebb0092ca0145adb5 P 84783f594c914ad8ab166b8a0ac6daf7 [term 1 FOLLOWER]: Refusing update from remote peer b2e3251de3c044a98343877d8325a469: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:20:01.496411 31622 consensus_queue.cc:1035] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84783f594c914ad8ab166b8a0ac6daf7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 43119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 06 16:20:01 dist-test-slave-t3q3 krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033200, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for HTTP/127.11.245.148@KRBTEST.COM
I20260806 16:20:01.510929 31641 mvcc.cc:205] Tried to move back new op lower bound from 7315591993258610688 to 7315591992986042368. Current Snapshot: MvccSnapshot[applied={T|T < 7315591993258610688 or (T in {7315591993258610688})}]
I20260806 16:20:01.511682 31637 mvcc.cc:205] Tried to move back new op lower bound from 7315591993258610688 to 7315591992986042368. Current Snapshot: MvccSnapshot[applied={T|T < 7315591993258610688}]
I20260806 16:20:01.524636 31643 mvcc.cc:205] Tried to move back new op lower bound from 7315591993258610688 to 7315591992986042368. Current Snapshot: MvccSnapshot[applied={T|T < 7315591993258610688}]
I20260806 16:20:01.531769 30906 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:20:01.532084 30906 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:20:01.536243 30906 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96580ebc0f934b2db6991a8a0db698b9: Sending DeleteTablet for 3 replicas of tablet 707d037284d2466ebb0092ca0145adb5
I20260806 16:20:01.537829 31087 tablet_service.cc:1514] Processing DeleteTablet for tablet 707d037284d2466ebb0092ca0145adb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:20:01 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:46628
I20260806 16:20:01.539456 31487 tablet_service.cc:1514] Processing DeleteTablet for tablet 707d037284d2466ebb0092ca0145adb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:20:01 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:50382
I20260806 16:20:01.539443 31283 tablet_service.cc:1514] Processing DeleteTablet for tablet 707d037284d2466ebb0092ca0145adb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:20:01 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:48970
I20260806 16:20:01.540710 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 30962
I20260806 16:20:01.548251 31655 tablet_replica.cc:330] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469: stopping tablet replica
I20260806 16:20:01.548888 31655 raft_consensus.cc:2229] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:20:01.549625 31655 raft_consensus.cc:2258] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:20:01.554925 31655 ts_tablet_manager.cc:1905] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:20:01.567549 31655 ts_tablet_manager.cc:1918] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:20:01.567740 31655 log.cc:1196] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/707d037284d2466ebb0092ca0145adb5
I20260806 16:20:01.569520 31655 ts_tablet_manager.cc:1939] T 707d037284d2466ebb0092ca0145adb5 P b2e3251de3c044a98343877d8325a469: Deleting consensus metadata
I20260806 16:20:01.571920 30888 catalog_manager.cc:4841] TS b2e3251de3c044a98343877d8325a469 (127.11.245.131:39143): tablet 707d037284d2466ebb0092ca0145adb5 (table test-table [id=0c8bbe14b9914cbabcac266993f2aa10]) successfully deleted
I20260806 16:20:01.660678 31652 tablet_replica.cc:330] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4: stopping tablet replica
I20260806 16:20:01.661314 31652 raft_consensus.cc:2229] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:20:01.661880 31652 raft_consensus.cc:2258] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:20:01.666250 31652 ts_tablet_manager.cc:1905] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:20:01.684681 31652 ts_tablet_manager.cc:1918] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:20:01.684870 31652 log.cc:1196] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/707d037284d2466ebb0092ca0145adb5
I20260806 16:20:01.685294 31652 ts_tablet_manager.cc:1939] T 707d037284d2466ebb0092ca0145adb5 P cf996ba5c1d0485c9b59dd83e01baab4: Deleting consensus metadata
I20260806 16:20:01.687434 30888 catalog_manager.cc:4841] TS cf996ba5c1d0485c9b59dd83e01baab4 (127.11.245.130:36601): tablet 707d037284d2466ebb0092ca0145adb5 (table test-table [id=0c8bbe14b9914cbabcac266993f2aa10]) successfully deleted
W20260806 16:20:01.691135 30889 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:43119 (error 108)
I20260806 16:20:01.694950 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 31183
I20260806 16:20:01.720906 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:01.712948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:43119 (local address 127.0.0.1:46638)
0806 16:20:01.720466 (+ 7518us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:01.720678 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:43119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7305,"thread_start_us":837,"threads_started":1}
W20260806 16:20:01.721252 30889 catalog_manager.cc:4584] TS 84783f594c914ad8ab166b8a0ac6daf7 (127.11.245.129:43119): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 707d037284d2466ebb0092ca0145adb5: Network error: Client connection negotiation failed: client connection to 127.11.245.129:43119: connect: Connection refused (error 111)
I20260806 16:20:01.775343 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:01.774785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:43119 (local address 127.0.0.1:46650)
0806 16:20:01.775039 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:01.775163 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:43119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 16:20:01.776453 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:01.776107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:43119 (local address 127.0.0.1:46658)
0806 16:20:01.776279 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:01.776339 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:43119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 16:20:01.795728 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 31383
I20260806 16:20:01.837311 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:01.836834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:43119 (local address 127.0.0.1:46670)
0806 16:20:01.837056 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:01.837159 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:43119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 16:20:01.838378 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:01.838062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:43119 (local address 127.0.0.1:46678)
0806 16:20:01.838228 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:01.838286 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:43119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 16:20:01.846675 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 30863
I20260806 16:20:01.863801 12246 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 16:20:02.232661 12246 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 16:20:02.232877 12246 mini_ranger.cc:66] Stopping Ranger...
I20260806 16:20:02.600693 12246 mini_ranger.cc:68] Stopped Ranger
2026-08-06 16:20:02.601 UTC [29574] LOG: received smart shutdown request
2026-08-06 16:20:02.606 UTC [29574] LOG: background worker "logical replication launcher" (PID 29591) exited with exit code 1
2026-08-06 16:20:02.607 UTC [29585] LOG: shutting down
2026-08-06 16:20:02.818 UTC [29574] LOG: database system is shut down
2026-08-06T16:20:02Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (59912 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:20:03 dist-test-slave-t3q3 krb5kdc[31670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:20:03 dist-test-slave-t3q3 krb5kdc[31670](info): set up 2 sockets
Aug 06 16:20:03 dist-test-slave-t3q3 krb5kdc[31670](info): commencing operation
krb5kdc: starting...
W20260806 16:20:04.270351 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.003s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:20:04 dist-test-slave-t3q3 krb5kdc[31670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:20:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:20:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:20:04.429423 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:39965
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:42995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:39965,127.11.245.189:37663,127.11.245.188:36991
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:20:04.576205 31690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:04.576574 31690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:04.576620 31690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:04.581008 31690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:20:04.581086 31690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:04.581106 31690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:04.581123 31690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:20:04.581169 31690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:20:04.587026 31690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:42995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:39965,127.11.245.189:37663,127.11.245.188:36991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:39965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:04.588570 31690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:04.590046 31690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:04.601629 31696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:04.601629 31700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:04.601637 31698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:04.602033 31690 server_base.cc:1034] running on GCE node
I20260806 16:20:04.602957 31690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:04.604532 31690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:04.605734 31690 hybrid_clock.cc:648] HybridClock initialized: now 1786033204605717 us; error 36 us; skew 500 ppm
Aug 06 16:20:04 dist-test-slave-t3q3 krb5kdc[31670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033204, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:04.609290 31690 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:20:04.610548 31690 webserver.cc:460] Webserver started at http://127.11.245.190:36211/ using document root <none> and password file <none>
I20260806 16:20:04.611281 31690 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:04.611354 31690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:04.611581 31690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:20:04.614195 31690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "11c96ef59f9e4abfb1ac28141c65e7d8"
format_stamp: "Formatted at 2026-08-06 16:20:04 on dist-test-slave-t3q3"
server_key: "c772ae4c8112be82dc56c63e5ff706a5"
server_key_iv: "9c4823ccd202f18c6672cd29d763aad0"
server_key_version: "encryptionkey@0"
I20260806 16:20:04.614831 31690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "11c96ef59f9e4abfb1ac28141c65e7d8"
format_stamp: "Formatted at 2026-08-06 16:20:04 on dist-test-slave-t3q3"
server_key: "c772ae4c8112be82dc56c63e5ff706a5"
server_key_iv: "9c4823ccd202f18c6672cd29d763aad0"
server_key_version: "encryptionkey@0"
I20260806 16:20:04.619521 31690 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260806 16:20:04.623231 31709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:04.625142 31690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:20:04.625312 31690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "11c96ef59f9e4abfb1ac28141c65e7d8"
format_stamp: "Formatted at 2026-08-06 16:20:04 on dist-test-slave-t3q3"
server_key: "c772ae4c8112be82dc56c63e5ff706a5"
server_key_iv: "9c4823ccd202f18c6672cd29d763aad0"
server_key_version: "encryptionkey@0"
I20260806 16:20:04.625496 31690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:04.647145 31690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:04.651337 31690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:04.651636 31690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:04.660825 31690 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:39965
I20260806 16:20:04.661201 31780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:39965 every 8 connection(s)
I20260806 16:20:04.662175 31690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:20:04.666034 31781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:04.670830 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 31690
I20260806 16:20:04.671042 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:20:04.671399 12246 external_mini_cluster.cc:1428] Setting key ed588466ab3894a8f67cec1475dd2c8f
I20260806 16:20:04.671206 31781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.245.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.189@KRBTEST.COM" created.
Aug 06 16:20:04 dist-test-slave-t3q3 krb5kdc[31670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033204, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:04.696805 31787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:04.675237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44534 (local address 127.11.245.190:39965)
0806 16:20:04.675946 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:04.675974 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:04.676317 (+ 343us) server_negotiation.cc:407] Connection header received
0806 16:20:04.677724 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:04.677733 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:04.678034 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:04.678285 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:04.679743 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:04.681510 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:04.682854 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:04.683604 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:04.683866 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:04.689779 (+ 5913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:04.689789 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:04.689807 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:04.689840 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:04.692452 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:04.693111 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:04.693120 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:04.693122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:04.693316 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:04.693689 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:04.693695 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:04.693700 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:04.694446 (+ 746us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:04.694617 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:04.695129 (+ 512us) server_negotiation.cc:299] Negotiation successful
0806 16:20:04.695624 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":180,"server-negotiator.queue_time_us":480,"thread_start_us":211,"threads_started":1}
I20260806 16:20:04.696801 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:04.675237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:39965 (local address 127.0.0.1:44534)
0806 16:20:04.676096 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:04.676160 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:04.677312 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:04.678356 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:04.678444 (+ 88us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:04.679157 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:04.679516 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:04.679534 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:04.681697 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:04.681700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:04.682700 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:04.682716 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:04.683847 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:04.683851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:04.683978 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:04.684856 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:04.684880 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:04.689478 (+ 4598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:04.692630 (+ 3152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:04.692642 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:04.692656 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:04.692952 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:04.693428 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:04.693430 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:04.693431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:04.693587 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:04.694605 (+ 1018us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:04.694612 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:04.694784 (+ 172us) client_negotiation.cc:769] Sending connection context
0806 16:20:04.695014 (+ 230us) client_negotiation.cc:240] Negotiation successful
0806 16:20:04.695507 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":52,"spinlock_wait_cycles":73088,"thread_start_us":211,"threads_started":1}
I20260806 16:20:04.699963 31781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:20:04.701283 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:04.700924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37663 (local address 127.0.0.1:37052)
0806 16:20:04.701107 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:04.701200 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260806 16:20:04.701576 31713 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.245.189:37663: connect: Connection refused (error 111)
I20260806 16:20:04.702694 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:04.702414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37663 (local address 127.0.0.1:37068)
0806 16:20:04.702579 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:04.702632 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 16:20:04.702909 31781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.189:37663: Network error: Client connection negotiation failed: client connection to 127.11.245.189:37663: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.189.keytab.
Entry for principal kudu/127.11.245.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.189.keytab.
I20260806 16:20:04.712898 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.189:37663
--webserver_interface=127.11.245.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.189.keytab
--principal=kudu/127.11.245.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:42995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:39965,127.11.245.189:37663,127.11.245.188:36991
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:20:04.752223 31781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } attempt: 1
I20260806 16:20:04.753545 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:04.753140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37663 (local address 127.0.0.1:37076)
0806 16:20:04.753349 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:04.753432 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 16:20:04.754420 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:04.754168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37663 (local address 127.0.0.1:37090)
0806 16:20:04.754292 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:04.754353 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 16:20:04.754570 31781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.189:37663: Network error: Client connection negotiation failed: client connection to 127.11.245.189:37663: connect: Connection refused (error 111)
I20260806 16:20:04.822813 31781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } attempt: 2
I20260806 16:20:04.824247 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:04.823766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37663 (local address 127.0.0.1:37100)
0806 16:20:04.823975 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:04.824119 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 16:20:04.825150 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:04.824886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37663 (local address 127.0.0.1:37106)
0806 16:20:04.825018 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:04.825078 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 16:20:04.825346 31781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.189:37663: Network error: Client connection negotiation failed: client connection to 127.11.245.189:37663: connect: Connection refused (error 111)
W20260806 16:20:04.873221 31792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:04.873585 31792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:04.873632 31792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:04.878428 31792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:20:04.878542 31792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:04.878564 31792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:04.878587 31792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:20:04.878603 31792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:20:04.885185 31792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:42995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:39965,127.11.245.189:37663,127.11.245.188:36991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.189.keytab
--principal=kudu/127.11.245.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.189:37663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.245.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:04.886801 31792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:04.888254 31792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:20:04.899678 31792 server_base.cc:1034] running on GCE node
W20260806 16:20:04.899617 31799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:04.899617 31801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:04.899617 31803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:04.900769 31792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:04.902275 31792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:04.903509 31792 hybrid_clock.cc:648] HybridClock initialized: now 1786033204903492 us; error 32 us; skew 500 ppm
Aug 06 16:20:04 dist-test-slave-t3q3 krb5kdc[31670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033204, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:04.913617 31792 init.cc:377] Logged in from keytab as kudu/127.11.245.189@KRBTEST.COM (short username kudu)
I20260806 16:20:04.914949 31792 webserver.cc:460] Webserver started at http://127.11.245.189:40029/ using document root <none> and password file <none>
I20260806 16:20:04.915740 31792 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:04.915843 31792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:04.916546 31792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:20:04.916716 31781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } attempt: 3
I20260806 16:20:04.917867 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:04.917460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37663 (local address 127.0.0.1:37118)
0806 16:20:04.917671 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:04.917755 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 16:20:04.918979 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:04.918681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37663 (local address 127.0.0.1:37126)
0806 16:20:04.918815 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:04.918897 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 16:20:04.919174 31781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.189:37663: Network error: Client connection negotiation failed: client connection to 127.11.245.189:37663: connect: Connection refused (error 111)
I20260806 16:20:04.919739 31792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/data/instance:
uuid: "fd75be263fbf4056bd790630385b4b66"
format_stamp: "Formatted at 2026-08-06 16:20:04 on dist-test-slave-t3q3"
server_key: "6554a91b498c447f74d5bc10b740870f"
server_key_iv: "ed663ea7b7d7e6091fd8de70a8ca473d"
server_key_version: "encryptionkey@0"
I20260806 16:20:04.920545 31792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/wal/instance:
uuid: "fd75be263fbf4056bd790630385b4b66"
format_stamp: "Formatted at 2026-08-06 16:20:04 on dist-test-slave-t3q3"
server_key: "6554a91b498c447f74d5bc10b740870f"
server_key_iv: "ed663ea7b7d7e6091fd8de70a8ca473d"
server_key_version: "encryptionkey@0"
I20260806 16:20:04.925736 31792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 16:20:04.929155 31815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:04.931064 31792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 16:20:04.931249 31792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/wal
uuid: "fd75be263fbf4056bd790630385b4b66"
format_stamp: "Formatted at 2026-08-06 16:20:04 on dist-test-slave-t3q3"
server_key: "6554a91b498c447f74d5bc10b740870f"
server_key_iv: "ed663ea7b7d7e6091fd8de70a8ca473d"
server_key_version: "encryptionkey@0"
I20260806 16:20:04.931396 31792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:04.952687 31792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:04.956825 31792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:04.957108 31792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:04.966336 31792 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.189:37663
I20260806 16:20:04.966698 31882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.189:37663 every 8 connection(s)
I20260806 16:20:04.967671 31792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/data/info.pb
I20260806 16:20:04.971575 31883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:04.975049 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 31792
I20260806 16:20:04.975236 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-1/wal/instance
I20260806 16:20:04.975625 12246 external_mini_cluster.cc:1428] Setting key 4f7e833163a66e555eff963a9d6aad25
I20260806 16:20:04.978425 31883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.245.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.188@KRBTEST.COM" created.
Aug 06 16:20:04 dist-test-slave-t3q3 krb5kdc[31670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033204, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.189@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:04.996021 31787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:04.981346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44542 (local address 127.11.245.190:39965)
0806 16:20:04.981538 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:04.981542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:04.982942 (+ 1400us) server_negotiation.cc:407] Connection header received
0806 16:20:04.984144 (+ 1202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:04.984148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:04.984234 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:04.984344 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:04.985834 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:04.986423 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:04.987836 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:04.988248 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:04.988379 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:04.991390 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:04.991404 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:04.991410 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:04.991452 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:04.993211 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:04.993964 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:04.993968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:04.993969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:04.994058 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:04.994525 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:04.994528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:04.994530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:04.994764 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:04.994849 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:04.995697 (+ 848us) server_negotiation.cc:299] Negotiation successful
0806 16:20:04.995851 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 16:20:04.997587 31888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:04.981912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:39965 (local address 127.0.0.1:44542)
0806 16:20:04.982709 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:04.982785 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:04.983844 (+ 1059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:04.984568 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:04.984576 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:04.985009 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:04.985653 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:04.985669 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:04.986577 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:04.986581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:04.987682 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:04.987692 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:04.988446 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:04.988451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:04.988647 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:04.989410 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:04.989431 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:04.991183 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:04.993370 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:04.993378 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:04.993401 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:04.993823 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:04.994179 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:04.994184 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:04.994186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:04.994413 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:04.994896 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:04.994906 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:04.995341 (+ 435us) client_negotiation.cc:769] Sending connection context
0806 16:20:04.995598 (+ 257us) client_negotiation.cc:240] Negotiation successful
0806 16:20:04.996107 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":218,"threads_started":1}
I20260806 16:20:04.999476 31883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:20:05 dist-test-slave-t3q3 krb5kdc[31670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033204, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.189@KRBTEST.COM for kudu/127.11.245.189@KRBTEST.COM
I20260806 16:20:05.010696 31888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.000245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37663 (local address 127.0.0.1:37138)
0806 16:20:05.000383 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.000397 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:05.000476 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:05.001250 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:05.001253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:05.001462 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:05.001583 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:05.001590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.002727 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.002732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:05.003337 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:05.003346 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.004053 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.004057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:05.004181 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.004736 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:05.004748 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:05.006460 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:05.008874 (+ 2414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:05.008877 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:05.008880 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:05.009086 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:05.009417 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:05.009421 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:05.009422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:05.009506 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:05.010064 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:05.010067 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:05.010171 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 16:20:05.010294 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 16:20:05.010492 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 16:20:05.010855 31891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.000440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37138 (local address 127.11.245.189:37663)
0806 16:20:05.000853 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:05.000864 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:05.000885 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:20:05.000921 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:05.000926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:05.001115 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:05.001281 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:05.001702 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.002592 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.003461 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.003922 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.004077 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.006618 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:05.006629 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:05.006645 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:05.006668 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:05.008717 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.009209 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.009218 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.009224 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.009301 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.009604 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.009607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.009609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.009899 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:05.010073 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:05.010441 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 16:20:05.010653 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":203,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.188.keytab.
Entry for principal kudu/127.11.245.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.188.keytab.
I20260806 16:20:05.012919 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.188:36991
--webserver_interface=127.11.245.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.188.keytab
--principal=kudu/127.11.245.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:42995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:39965,127.11.245.189:37663,127.11.245.188:36991
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:20:05.014138 31883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:20:05.015131 31888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.014786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39554)
0806 16:20:05.014948 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.015034 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 16:20:05.015396 31817 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
I20260806 16:20:05.016552 31888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.016271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39564)
0806 16:20:05.016407 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.016471 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 16:20:05.016739 31883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:36991: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
I20260806 16:20:05.062433 31781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } attempt: 4
I20260806 16:20:05.066094 31883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } attempt: 1
I20260806 16:20:05.067162 31888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.066742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39574)
0806 16:20:05.066959 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.067050 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 16:20:05.068776 31888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.068416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39590)
0806 16:20:05.068601 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.068673 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260806 16:20:05.069059 31883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:36991: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
Aug 06 16:20:05 dist-test-slave-t3q3 krb5kdc[31670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033204, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for kudu/127.11.245.189@KRBTEST.COM
I20260806 16:20:05.074471 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.063475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37663 (local address 127.0.0.1:37144)
0806 16:20:05.063722 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.063745 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:05.063883 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:05.064352 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:05.064356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:05.064595 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:05.064767 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:05.064774 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.066072 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.066077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:05.066862 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:05.066879 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.067655 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.067659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:05.067766 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.068530 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:05.068549 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:05.070377 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:05.072329 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:05.072333 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:05.072336 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:05.072618 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:05.073064 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:05.073068 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:05.073071 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:05.073186 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:05.073709 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:05.073713 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:05.073856 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 16:20:05.073978 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 16:20:05.074165 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260806 16:20:05.074510 31891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.063592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37144 (local address 127.11.245.189:37663)
0806 16:20:05.063770 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:05.063776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:05.063827 (+ 51us) server_negotiation.cc:407] Connection header received
0806 16:20:05.064014 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:05.064019 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:05.064123 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:05.064240 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:05.064953 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.065839 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.067022 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.067487 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.067640 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.070580 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:05.070585 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:05.070588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:05.070616 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:05.072208 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.072753 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.072757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.072758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.072862 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.073330 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.073334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.073336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.073593 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:05.073694 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:05.074059 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 16:20:05.074202 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 16:20:05.076256 31781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:20:05.077610 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.077294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39594)
0806 16:20:05.077453 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.077526 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 16:20:05.078537 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.078283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39602)
0806 16:20:05.078405 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.078463 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 16:20:05.078701 31781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:36991: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
I20260806 16:20:05.137362 31883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } attempt: 2
I20260806 16:20:05.137856 31781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } attempt: 1
I20260806 16:20:05.138530 31888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.138038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39606)
0806 16:20:05.138283 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.138390 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260806 16:20:05.138973 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.138598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39618)
0806 16:20:05.138778 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.138870 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 16:20:05.139695 31888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.139414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39622)
0806 16:20:05.139559 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.139621 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 16:20:05.139829 31883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:36991: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
I20260806 16:20:05.140020 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.139619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39630)
0806 16:20:05.139760 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.139945 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 16:20:05.140203 31781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:36991: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
W20260806 16:20:05.169479 31892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:05.169893 31892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:05.169974 31892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:05.174583 31892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:20:05.174701 31892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:05.174741 31892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:05.174785 31892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:20:05.174827 31892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:20:05.181344 31892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:42995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:39965,127.11.245.189:37663,127.11.245.188:36991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.188.keytab
--principal=kudu/127.11.245.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.188:36991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.245.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:05.183022 31892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:05.184508 31892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:05.195214 31906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:05.195196 31902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:05.195576 31901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:05.196332 31892 server_base.cc:1034] running on GCE node
I20260806 16:20:05.197108 31892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:05.198827 31892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:05.200145 31892 hybrid_clock.cc:648] HybridClock initialized: now 1786033205200111 us; error 50 us; skew 500 ppm
Aug 06 16:20:05 dist-test-slave-t3q3 krb5kdc[31670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033205, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:05.203790 31892 init.cc:377] Logged in from keytab as kudu/127.11.245.188@KRBTEST.COM (short username kudu)
I20260806 16:20:05.205155 31892 webserver.cc:460] Webserver started at http://127.11.245.188:33799/ using document root <none> and password file <none>
I20260806 16:20:05.205938 31892 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:05.206048 31892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:05.206362 31892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:20:05.207407 31781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } attempt: 2
I20260806 16:20:05.208649 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.208226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39642)
0806 16:20:05.208479 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.208549 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260806 16:20:05.209483 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.209220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39650)
0806 16:20:05.209346 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.209409 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 16:20:05.209621 31781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:36991: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
I20260806 16:20:05.209618 31892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/data/instance:
uuid: "180177bce01f433db3029230561dfa47"
format_stamp: "Formatted at 2026-08-06 16:20:05 on dist-test-slave-t3q3"
server_key: "36c3235198fe073d02e737ba8243f104"
server_key_iv: "361b252f1b717490462c676c4f0ea221"
server_key_version: "encryptionkey@0"
I20260806 16:20:05.210381 31892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/wal/instance:
uuid: "180177bce01f433db3029230561dfa47"
format_stamp: "Formatted at 2026-08-06 16:20:05 on dist-test-slave-t3q3"
server_key: "36c3235198fe073d02e737ba8243f104"
server_key_iv: "361b252f1b717490462c676c4f0ea221"
server_key_version: "encryptionkey@0"
I20260806 16:20:05.215104 31892 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 16:20:05.218241 31922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:05.219900 31892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:20:05.220166 31892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/wal
uuid: "180177bce01f433db3029230561dfa47"
format_stamp: "Formatted at 2026-08-06 16:20:05 on dist-test-slave-t3q3"
server_key: "36c3235198fe073d02e737ba8243f104"
server_key_iv: "361b252f1b717490462c676c4f0ea221"
server_key_version: "encryptionkey@0"
I20260806 16:20:05.220319 31892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:05.231040 31883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } attempt: 3
I20260806 16:20:05.232199 31888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.231796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39660)
0806 16:20:05.231992 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.232090 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 16:20:05.233325 31888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.232855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39672)
0806 16:20:05.233185 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.233244 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243}
W20260806 16:20:05.233556 31883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:36991: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36991: connect: Connection refused (error 111)
I20260806 16:20:05.243755 31892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:05.248389 31892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:05.248668 31892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:05.258574 31892 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.188:36991
I20260806 16:20:05.258890 31987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.188:36991 every 8 connection(s)
I20260806 16:20:05.259959 31892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/data/info.pb
I20260806 16:20:05.264037 31988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:05.264607 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 31892
I20260806 16:20:05.264818 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786033011181791-12246-0/minicluster-data/master-2/wal/instance
I20260806 16:20:05.265264 12246 external_mini_cluster.cc:1428] Setting key 1ce9097bb2d42d1728cd1d90a869db2e
I20260806 16:20:05.270813 31988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:20:05 dist-test-slave-t3q3 krb5kdc[31670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.188@KRBTEST.COM
Aug 06 16:20:05 dist-test-slave-t3q3 krb5kdc[31670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033205, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.188@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:05.285509 31992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.267103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39688 (local address 127.11.245.188:36991)
0806 16:20:05.267626 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:05.267647 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:05.267696 (+ 49us) server_negotiation.cc:407] Connection header received
0806 16:20:05.268615 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:05.268642 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:05.269076 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:05.269432 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:05.270565 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.272022 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.273236 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.274167 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.274904 (+ 737us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.278731 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:05.278739 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:05.278754 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:05.278788 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:05.281177 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.281706 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.281713 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.281722 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.281891 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.282223 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.282227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.282229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.282781 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:05.283011 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:05.283408 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 16:20:05.283850 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":176,"threads_started":1}
I20260806 16:20:05.285938 31996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.273303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:39965 (local address 127.0.0.1:44544)
0806 16:20:05.273746 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.273796 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:05.273986 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:05.274342 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:05.274349 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:05.274965 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:05.275181 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:05.275189 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.276379 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.276384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:05.277094 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:05.277104 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.277782 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.277785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:05.277880 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.278762 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:05.278847 (+ 85us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:05.281181 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:05.283218 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:05.283229 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:05.283237 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:05.283637 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:05.284119 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:05.284124 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:05.284126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:05.284431 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:05.285062 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:05.285070 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:05.285220 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 16:20:05.285486 (+ 266us) client_negotiation.cc:240] Negotiation successful
0806 16:20:05.285757 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"spinlock_wait_cycles":1920,"thread_start_us":132,"threads_started":1}
I20260806 16:20:05.286003 31787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.273350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44544 (local address 127.11.245.190:39965)
0806 16:20:05.273525 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:05.273532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:05.273907 (+ 375us) server_negotiation.cc:407] Connection header received
0806 16:20:05.274105 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:05.274108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:05.274176 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:05.274277 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:05.275352 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.276231 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.277230 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.277577 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.277709 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.281361 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:05.281365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:05.281367 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:05.281401 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:05.283051 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.283784 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.283791 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.283802 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.283914 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.284546 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.284550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.284552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.284882 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:05.285000 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:05.285578 (+ 578us) server_negotiation.cc:299] Negotiation successful
0806 16:20:05.285768 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 16:20:05.288563 31988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:20:05 dist-test-slave-t3q3 krb5kdc[31670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033205, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.188@KRBTEST.COM for kudu/127.11.245.189@KRBTEST.COM
Aug 06 16:20:05 dist-test-slave-t3q3 krb5kdc[31670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:05.299552 31996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.289360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:37663 (local address 127.0.0.1:37156)
0806 16:20:05.289833 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.289847 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:05.289935 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:05.290239 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:05.290242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:05.290408 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:05.290545 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:05.290550 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.291419 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.291424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:05.292052 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:05.292115 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.292667 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.292672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:05.292793 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.293556 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:05.293569 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:05.295346 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:05.297482 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:05.297486 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:05.297489 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:05.297759 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:05.298143 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:05.298146 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:05.298149 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:05.298264 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:05.298857 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:05.298861 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:05.299037 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 16:20:05.299152 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 16:20:05.299329 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389}
I20260806 16:20:05.299589 31891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.289512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37156 (local address 127.11.245.189:37663)
0806 16:20:05.289679 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:05.289688 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:05.289930 (+ 242us) server_negotiation.cc:407] Connection header received
0806 16:20:05.290035 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:05.290038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:05.290108 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:05.290214 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:05.290684 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.291294 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.292218 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.292542 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.292633 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.295529 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:05.295537 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:05.295541 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:05.295576 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:05.297334 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.297910 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.297913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.297915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.298010 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.298405 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.298412 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.298416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.298741 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:05.298833 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:05.299272 (+ 439us) server_negotiation.cc:299] Negotiation successful
0806 16:20:05.299452 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 16:20:05.300758 31787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.289595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.11.245.190:39965)
0806 16:20:05.289737 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:05.289744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:05.289760 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:20:05.289800 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:05.289803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:05.289858 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:05.289943 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:05.290840 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.291468 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.292402 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.292659 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.292769 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.295917 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:05.295922 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:05.295926 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:05.295959 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:05.298134 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.298836 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.298841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.298842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.298949 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.299336 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.299340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.299341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.299630 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:05.299937 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:05.300388 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 16:20:05.300559 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 16:20:05.300973 31988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:20:05 dist-test-slave-t3q3 krb5kdc[31670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033205, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.188@KRBTEST.COM for kudu/127.11.245.188@KRBTEST.COM
Aug 06 16:20:05 dist-test-slave-t3q3 krb5kdc[31670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.189@KRBTEST.COM
I20260806 16:20:05.309815 31781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } attempt: 3
I20260806 16:20:05.310250 31992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.301970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39702 (local address 127.11.245.188:36991)
0806 16:20:05.302094 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:05.302099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:05.302113 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:20:05.302156 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:05.302159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:05.302207 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:05.302289 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:05.302839 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.303400 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.304209 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.304453 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.304570 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.306840 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:05.306849 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:05.306851 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:05.306879 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:05.308564 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.308982 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.308986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.308988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.309065 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.309352 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.309355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.309357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.309554 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:05.309660 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:05.309934 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 16:20:05.310079 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":49}
I20260806 16:20:05.310250 31996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.301666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39702)
0806 16:20:05.301849 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.301865 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:05.302030 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:05.302404 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:05.302409 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:05.302633 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:05.302741 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:05.302750 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.303517 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.303521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:05.304103 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:05.304117 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.304566 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.304569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:05.304690 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.305278 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:05.305291 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:05.306699 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:05.308674 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:05.308679 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:05.308681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:05.308880 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:05.309163 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:05.309166 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:05.309168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:05.309246 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:05.309658 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:05.309661 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:05.309751 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 16:20:05.309858 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 16:20:05.310026 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":22}
I20260806 16:20:05.312249 31891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.302217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37168 (local address 127.11.245.189:37663)
0806 16:20:05.302340 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:05.302344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:05.302390 (+ 46us) server_negotiation.cc:407] Connection header received
0806 16:20:05.302534 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:05.302537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:05.302590 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:05.302680 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:05.303408 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.303960 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.304828 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.305218 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.305343 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.308183 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:05.308186 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:05.308189 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:05.308214 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:05.309781 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.310318 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.310322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.310325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.310420 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.310733 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.310739 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.310743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.311001 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:05.311313 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:05.311707 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 16:20:05.311844 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 16:20:05.315936 31988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47: Bootstrap starting.
Aug 06 16:20:05 dist-test-slave-t3q3 krb5kdc[31670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033204, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for kudu/127.11.245.188@KRBTEST.COM
I20260806 16:20:05.316701 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:20:05.320160 31988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:05.321117 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.310490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39712)
0806 16:20:05.310852 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.310870 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:05.310962 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:05.311306 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:05.311310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:05.311540 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:05.311683 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:05.311690 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.312775 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.312782 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:05.313549 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:05.313561 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.314152 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.314158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:05.314271 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.315133 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:05.315150 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:05.317003 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:05.319028 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:05.319032 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:05.319035 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:05.319332 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:05.319715 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:05.319718 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:05.319720 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:05.319832 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:05.320458 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:05.320462 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:05.320617 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 16:20:05.320703 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 16:20:05.320885 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251}
I20260806 16:20:05.321579 31988 log.cc:824] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:05.322203 31992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.310793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39712 (local address 127.11.245.188:36991)
0806 16:20:05.310902 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:05.310907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:05.310919 (+ 12us) server_negotiation.cc:407] Connection header received
0806 16:20:05.311087 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:05.311091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:05.311164 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:05.311799 (+ 635us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:05.311897 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.312628 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.313708 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.314010 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.314827 (+ 817us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.317154 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:05.317159 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:05.317161 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:05.317188 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:05.318882 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.319464 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.319468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.319470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.319581 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.319964 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.319966 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.319968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.320323 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:05.321815 (+ 1492us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:05.321899 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 16:20:05.322040 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 16:20:05.324700 31988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47: No bootstrap required, opened a new log
I20260806 16:20:05.326961 31781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8: Bootstrap starting.
I20260806 16:20:05.330667 31787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.319963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44572 (local address 127.11.245.190:39965)
0806 16:20:05.320146 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:05.320151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:05.320418 (+ 267us) server_negotiation.cc:407] Connection header received
0806 16:20:05.320624 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:05.320627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:05.320679 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:05.320733 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:05.321832 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.322656 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.323847 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.324182 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.324274 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.326621 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:05.326625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:05.326629 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:05.326659 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:05.328448 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.328980 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.328991 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.328992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.329118 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.329522 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.329525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.329527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.329789 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:05.329899 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:05.330322 (+ 423us) server_negotiation.cc:299] Negotiation successful
0806 16:20:05.330482 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 16:20:05.331106 31988 raft_consensus.cc:348] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } }
I20260806 16:20:05.331609 31781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:05.331666 31988 raft_consensus.cc:374] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:05.331800 31988 raft_consensus.cc:729] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 180177bce01f433db3029230561dfa47, State: Initialized, Role: FOLLOWER
I20260806 16:20:05.332679 31988 consensus_queue.cc:260] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } }
I20260806 16:20:05.332850 31781 log.cc:824] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:05.333201 31891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.319710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37176 (local address 127.11.245.189:37663)
0806 16:20:05.319870 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:05.319876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:05.321348 (+ 1472us) server_negotiation.cc:407] Connection header received
0806 16:20:05.321581 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:05.321584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:05.321654 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:05.321715 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:05.322802 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.323519 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.324720 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.325060 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.325159 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.327832 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:05.327836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:05.327839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:05.327876 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:05.329784 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.330404 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.330408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.330409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.330529 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.331614 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.331618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.331619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.331902 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:05.332050 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:05.332845 (+ 795us) server_negotiation.cc:299] Negotiation successful
0806 16:20:05.333000 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 16:20:05.333720 32012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } } }
I20260806 16:20:05.334013 32012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:20:05.334806 31988 sys_catalog.cc:564] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:20:05.336334 31781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8: No bootstrap required, opened a new log
I20260806 16:20:05.342995 31781 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } }
I20260806 16:20:05.343365 31781 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:05.343443 31781 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c96ef59f9e4abfb1ac28141c65e7d8, State: Initialized, Role: FOLLOWER
I20260806 16:20:05.344134 31781 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } }
I20260806 16:20:05.340659 31892 master_runner.cc:186] Error getting master registration for 127.11.245.190:39965: OK, instance_id { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" instance_seqno: 1786033204658267 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:20:05.344647 31892 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:20:05.344981 32030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } } }
I20260806 16:20:05.345183 32030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:20:05.346045 31781 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 16:20:05.348212 32029 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 16:20:05.348278 32007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.320619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39714 (local address 127.11.245.188:36991)
0806 16:20:05.330815 (+ 10196us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:05.330825 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:05.332105 (+ 1280us) server_negotiation.cc:407] Connection header received
0806 16:20:05.332160 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:05.332163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:05.332230 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:05.332325 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:05.337361 (+ 5036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.338384 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.339726 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.340145 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.340296 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.342690 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:05.342697 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:05.342700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:05.342729 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:05.345341 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.346313 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.346317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.346319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.346463 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.346868 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.346872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.346873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.347172 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:05.347319 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:05.347811 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 16:20:05.347986 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10091,"thread_start_us":111,"threads_started":1}
W20260806 16:20:05.348333 32029 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 16:20:05.353035 32046 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:20:05.353140 32046 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:20:05.352595 31690 master_runner.cc:186] Error getting master registration for 127.11.245.189:37663: OK, instance_id { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" instance_seqno: 1786033204963708 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:20:05.357699 31690 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:20:05.376744 31883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } attempt: 4
I20260806 16:20:05.378787 32030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:20:05.379027 32030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } }
I20260806 16:20:05.380525 32030 leader_election.cc:288] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd75be263fbf4056bd790630385b4b66 (127.11.245.189:37663), 180177bce01f433db3029230561dfa47 (127.11.245.188:36991)
I20260806 16:20:05.381996 31953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "180177bce01f433db3029230561dfa47" is_pre_election: true
I20260806 16:20:05.382454 31953 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11c96ef59f9e4abfb1ac28141c65e7d8 in term 0.
I20260806 16:20:05.383289 31711 leader_election.cc:302] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c96ef59f9e4abfb1ac28141c65e7d8, 180177bce01f433db3029230561dfa47; no voters:
Aug 06 16:20:05 dist-test-slave-t3q3 krb5kdc[31670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033204, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.189@KRBTEST.COM for kudu/127.11.245.188@KRBTEST.COM
I20260806 16:20:05.383878 32030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:20:05.383978 32030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:20:05.384025 32030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:05.385917 32030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } }
I20260806 16:20:05.387863 31953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "180177bce01f433db3029230561dfa47"
I20260806 16:20:05.388034 31953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:05.388564 31888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.377454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36991 (local address 127.0.0.1:39722)
0806 16:20:05.377646 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:05.377666 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:05.377768 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:05.378207 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:05.378210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:05.378514 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:05.378657 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:05.378665 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.379845 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.379850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:05.380781 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:05.380794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.381604 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.381608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:05.381708 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.382379 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:05.382395 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:05.384213 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:05.386486 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:05.386490 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:05.386504 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:05.386865 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:05.387227 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:05.387230 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:05.387232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:05.387334 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:05.387913 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:05.387916 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:05.388033 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 16:20:05.388165 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 16:20:05.388355 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 16:20:05.389106 32007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:05.377791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39722 (local address 127.11.245.188:36991)
0806 16:20:05.377932 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:05.377937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:05.377954 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:20:05.378006 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:05.378009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:05.378062 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:05.378161 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:05.378811 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.379701 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.380929 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:05.381331 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:05.381436 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:05.384390 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:05.384395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:05.384398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:05.384428 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:05.386331 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.387006 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.387010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.387012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.387116 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:05.387451 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:05.387455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:05.387456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:05.387759 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:05.387860 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:05.388789 (+ 929us) server_negotiation.cc:299] Negotiation successful
0806 16:20:05.388947 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:20:05.389845 31953 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11c96ef59f9e4abfb1ac28141c65e7d8 in term 1.
I20260806 16:20:05.391968 32030 leader_election.cc:288] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [CANDIDATE]: Term 1 election: Requested vote from peers fd75be263fbf4056bd790630385b4b66 (127.11.245.189:37663), 180177bce01f433db3029230561dfa47 (127.11.245.188:36991)
I20260806 16:20:05.392446 31711 leader_election.cc:302] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c96ef59f9e4abfb1ac28141c65e7d8, 180177bce01f433db3029230561dfa47; no voters:
I20260806 16:20:05.392765 32030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:20:05.393301 32030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [term 1 LEADER]: Becoming Leader. State: Replica: 11c96ef59f9e4abfb1ac28141c65e7d8, State: Running, Role: LEADER
I20260806 16:20:05.394150 32030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } }
I20260806 16:20:05.395816 31883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66: Bootstrap starting.
I20260806 16:20:05.402554 31883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:05.403743 32030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11c96ef59f9e4abfb1ac28141c65e7d8. Latest consensus state: current_term: 1 leader_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } } }
W20260806 16:20:05.404042 31716 tablet.cc:2369] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:20:05.404629 31883 log.cc:824] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:05.404888 32030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [sys.catalog]: This master's current role is: LEADER
I20260806 16:20:05.408839 32051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:20:05.410594 32051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:20:05.414304 31883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66: No bootstrap required, opened a new log
I20260806 16:20:05.423417 31953 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47 [term 1 FOLLOWER]: Refusing update from remote peer 11c96ef59f9e4abfb1ac28141c65e7d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:20:05.424505 32049 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:20:05.425524 31883 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } }
I20260806 16:20:05.426262 31883 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:05.426445 31883 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75be263fbf4056bd790630385b4b66, State: Initialized, Role: FOLLOWER
I20260806 16:20:05.427619 31883 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } }
I20260806 16:20:05.431087 32012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11c96ef59f9e4abfb1ac28141c65e7d8. Latest consensus state: current_term: 1 leader_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } } }
I20260806 16:20:05.431262 32012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:20:05.431640 32055 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } } }
I20260806 16:20:05.431815 32055 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:20:05.435768 31847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75be263fbf4056bd790630385b4b66"
I20260806 16:20:05.436051 31846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:05.436391 31849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75be263fbf4056bd790630385b4b66" is_pre_election: true
I20260806 16:20:05.437516 31883 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:20:05.437721 32049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } } }
I20260806 16:20:05.437882 32049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [sys.catalog]: This master's current role is: LEADER
I20260806 16:20:05.438037 31846 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66 [term 1 FOLLOWER]: Refusing update from remote peer 11c96ef59f9e4abfb1ac28141c65e7d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:20:05.438807 32030 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:20:05.439193 32012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } } }
I20260806 16:20:05.439365 32012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:20:05.439682 32054 mvcc.cc:205] Tried to move back new op lower bound from 7315592009401827328 to 7315592009302728704. Current Snapshot: MvccSnapshot[applied={T|T < 7315592009401827328}]
I20260806 16:20:05.441316 32057 mvcc.cc:205] Tried to move back new op lower bound from 7315592009401827328 to 7315592009302728704. Current Snapshot: MvccSnapshot[applied={T|T < 7315592009401827328}]
I20260806 16:20:05.445593 32055 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11c96ef59f9e4abfb1ac28141c65e7d8. Latest consensus state: current_term: 1 leader_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } } }
I20260806 16:20:05.445732 32055 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:20:05.445858 32055 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } } }
I20260806 16:20:05.445925 32055 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:20:05.448516 32079 mvcc.cc:205] Tried to move back new op lower bound from 7315592009401827328 to 7315592009302728704. Current Snapshot: MvccSnapshot[applied={T|T < 7315592009401827328}]
I20260806 16:20:05.450984 32051 catalog_manager.cc:1347] Generated new cluster ID: 759a799cff9a4cbba2d4669ce8a03c71
I20260806 16:20:05.451078 32051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260806 16:20:05.455687 32076 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:20:05.455833 32076 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:20:05.463887 32030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c96ef59f9e4abfb1ac28141c65e7d8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 39965 } } peers { permanent_uuid: "fd75be263fbf4056bd790630385b4b66" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 37663 } } peers { permanent_uuid: "180177bce01f433db3029230561dfa47" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36991 } } }
I20260806 16:20:05.464123 32030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8 [sys.catalog]: This master's current role is: LEADER
I20260806 16:20:05.471824 32051 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:20:05.473532 32051 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:20:05.491212 32051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11c96ef59f9e4abfb1ac28141c65e7d8: Generated new TSK 0
I20260806 16:20:05.492451 32051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:20:06.350281 32029 catalog_manager.cc:1259] Loaded cluster ID: 759a799cff9a4cbba2d4669ce8a03c71
I20260806 16:20:06.350404 32029 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47: loading cluster ID for follower catalog manager: success
I20260806 16:20:06.353192 32029 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47: acquiring CA information for follower catalog manager: success
I20260806 16:20:06.356333 32029 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 180177bce01f433db3029230561dfa47: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:20:06.457746 32076 catalog_manager.cc:1259] Loaded cluster ID: 759a799cff9a4cbba2d4669ce8a03c71
I20260806 16:20:06.457913 32076 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66: loading cluster ID for follower catalog manager: success
I20260806 16:20:06.460309 32076 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66: acquiring CA information for follower catalog manager: success
I20260806 16:20:06.462126 32076 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fd75be263fbf4056bd790630385b4b66: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:20:06.566053 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 31690
I20260806 16:20:06.588732 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 31792
I20260806 16:20:06.607064 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 31892
2026-08-06T16:20:06Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3650 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:20:06 dist-test-slave-t3q3 krb5kdc[32098](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:20:06 dist-test-slave-t3q3 krb5kdc[32098](info): set up 2 sockets
Aug 06 16:20:06 dist-test-slave-t3q3 krb5kdc[32098](info): commencing operation
krb5kdc: starting...
W20260806 16:20:07.979732 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.000s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:20:08 dist-test-slave-t3q3 krb5kdc[32098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:20:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:20:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:20:08.145434 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:33767
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:46231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:33767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:20:08.295042 32117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:08.295456 32117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:08.295504 32117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:08.300084 32117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:20:08.300189 32117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:08.300209 32117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:08.300231 32117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:20:08.300247 32117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:20:08.307195 32117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:46231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:33767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:33767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:08.308918 32117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:08.310379 32117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:20:08.322127 32117 server_base.cc:1034] running on GCE node
W20260806 16:20:08.322009 32126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:08.322007 32124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:08.322007 32123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:08.323189 32117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:08.325132 32117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:08.328450 32117 hybrid_clock.cc:648] HybridClock initialized: now 1786033208328253 us; error 203 us; skew 500 ppm
Aug 06 16:20:08 dist-test-slave-t3q3 krb5kdc[32098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033208, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:08.334144 32117 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:20:08.335870 32117 webserver.cc:460] Webserver started at http://127.11.245.190:38601/ using document root <none> and password file <none>
I20260806 16:20:08.336961 32117 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:08.337067 32117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:08.337440 32117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:20:08.341506 32117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "759bf308fed643a0a1345362f1f4886e"
format_stamp: "Formatted at 2026-08-06 16:20:08 on dist-test-slave-t3q3"
server_key: "3fb61591339f037a71e2d898e608478a"
server_key_iv: "9e6223b31e3644ae9da8a1681aa05bef"
server_key_version: "encryptionkey@0"
I20260806 16:20:08.342484 32117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "759bf308fed643a0a1345362f1f4886e"
format_stamp: "Formatted at 2026-08-06 16:20:08 on dist-test-slave-t3q3"
server_key: "3fb61591339f037a71e2d898e608478a"
server_key_iv: "9e6223b31e3644ae9da8a1681aa05bef"
server_key_version: "encryptionkey@0"
I20260806 16:20:08.349892 32117 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.002s
I20260806 16:20:08.353646 32133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:08.355813 32117 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260806 16:20:08.356046 32117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "759bf308fed643a0a1345362f1f4886e"
format_stamp: "Formatted at 2026-08-06 16:20:08 on dist-test-slave-t3q3"
server_key: "3fb61591339f037a71e2d898e608478a"
server_key_iv: "9e6223b31e3644ae9da8a1681aa05bef"
server_key_version: "encryptionkey@0"
I20260806 16:20:08.356271 32117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:08.384804 32117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:08.390213 32117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:08.390498 32117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:08.401594 32117 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:33767
I20260806 16:20:08.401628 32203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:33767 every 8 connection(s)
I20260806 16:20:08.403126 32117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:20:08.407410 32205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:08.409817 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 32117
I20260806 16:20:08.410025 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:20:08.410414 12246 external_mini_cluster.cc:1428] Setting key 159c3fbb19b529505bc8f2b2cc226da0
I20260806 16:20:08.417497 32205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e: Bootstrap starting.
I20260806 16:20:08.421093 32205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e: Neither blocks nor log segments found. Creating new log.
Aug 06 16:20:08 dist-test-slave-t3q3 krb5kdc[32098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:08.422209 32205 log.cc:824] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:08.425693 32205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e: No bootstrap required, opened a new log
I20260806 16:20:08.430065 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:08.412486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42196 (local address 127.11.245.190:33767)
0806 16:20:08.413143 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:08.413169 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:08.413217 (+ 48us) server_negotiation.cc:407] Connection header received
0806 16:20:08.414007 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:08.414031 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:08.414469 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:08.414917 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:08.415878 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:08.417077 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:08.418071 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:08.418616 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:08.418811 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:08.421885 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:08.421892 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:08.421912 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:08.421954 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:08.424729 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:08.425458 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:08.425470 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:08.425479 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:08.425687 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:08.426030 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:08.426035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:08.426037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:08.426784 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:08.427984 (+ 1200us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:08.428231 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 16:20:08.428598 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":202,"threads_started":1}
I20260806 16:20:08.430706 32205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759bf308fed643a0a1345362f1f4886e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33767 } }
I20260806 16:20:08.431047 32205 raft_consensus.cc:374] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:08.431155 32205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 759bf308fed643a0a1345362f1f4886e, State: Initialized, Role: FOLLOWER
I20260806 16:20:08.431800 32205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759bf308fed643a0a1345362f1f4886e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33767 } }
I20260806 16:20:08.432003 32205 raft_consensus.cc:388] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:20:08.432127 32205 raft_consensus.cc:482] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:20:08.432278 32205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:08.433650 32205 raft_consensus.cc:504] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759bf308fed643a0a1345362f1f4886e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33767 } }
I20260806 16:20:08.434147 32205 leader_election.cc:302] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 759bf308fed643a0a1345362f1f4886e; no voters:
I20260806 16:20:08.434648 32205 leader_election.cc:288] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:20:08.434755 32215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:20:08.435001 32215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [term 1 LEADER]: Becoming Leader. State: Replica: 759bf308fed643a0a1345362f1f4886e, State: Running, Role: LEADER
I20260806 16:20:08.435388 32215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759bf308fed643a0a1345362f1f4886e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33767 } }
I20260806 16:20:08.435648 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:20:08.436486 32205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:20:08.437510 32219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 759bf308fed643a0a1345362f1f4886e. Latest consensus state: current_term: 1 leader_uuid: "759bf308fed643a0a1345362f1f4886e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759bf308fed643a0a1345362f1f4886e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33767 } } }
I20260806 16:20:08.437707 32219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [sys.catalog]: This master's current role is: LEADER
I20260806 16:20:08.438122 32218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "759bf308fed643a0a1345362f1f4886e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759bf308fed643a0a1345362f1f4886e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 33767 } } }
I20260806 16:20:08.438235 32218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e [sys.catalog]: This master's current role is: LEADER
I20260806 16:20:08.438575 32228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:20:08.442595 32228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:20:08.450541 32228 catalog_manager.cc:1347] Generated new cluster ID: b0c21b104dc54185a786f4fd4d58b08e
I20260806 16:20:08.450667 32228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:20:08.463378 32228 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:20:08.464866 32228 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:20:08.476444 32228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 759bf308fed643a0a1345362f1f4886e: Generated new TSK 0
I20260806 16:20:08.477708 32228 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:20:08.494644 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 32117
2026-08-06T16:20:08Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1881 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:20:08 dist-test-slave-t3q3 krb5kdc[32244](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:20:08 dist-test-slave-t3q3 krb5kdc[32244](info): set up 2 sockets
Aug 06 16:20:08 dist-test-slave-t3q3 krb5kdc[32244](info): commencing operation
krb5kdc: starting...
W20260806 16:20:09.790270 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.003s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:20:09 dist-test-slave-t3q3 krb5kdc[32244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:20:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:20:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:20:09.961427 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:44261
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:44261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:20:10.112627 32269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:10.113134 32269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:10.113226 32269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:10.117726 32269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:20:10.117849 32269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:10.117897 32269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:10.117938 32269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:20:10.117974 32269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:20:10.124284 32269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:44261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:44261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:10.125922 32269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:10.127351 32269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:20:10.139007 32269 server_base.cc:1034] running on GCE node
W20260806 16:20:10.138978 32276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:10.138942 32282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:10.138942 32278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:10.140059 32269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:10.141896 32269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:10.143201 32269 hybrid_clock.cc:648] HybridClock initialized: now 1786033210143176 us; error 52 us; skew 500 ppm
Aug 06 16:20:10 dist-test-slave-t3q3 krb5kdc[32244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033210, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:10.152565 32269 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:20:10.154121 32269 webserver.cc:460] Webserver started at http://127.11.245.190:40297/ using document root <none> and password file <none>
I20260806 16:20:10.154915 32269 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:10.154990 32269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:10.155253 32269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:20:10.158157 32269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "16c771abcc6b41eea0cef489181e770c"
format_stamp: "Formatted at 2026-08-06 16:20:10 on dist-test-slave-t3q3"
server_key: "e33e7d4a1f5e2e692861cf7577b6dae0"
server_key_iv: "a1b2a56c423c2eaf703329eaea3923d6"
server_key_version: "encryptionkey@0"
I20260806 16:20:10.158839 32269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "16c771abcc6b41eea0cef489181e770c"
format_stamp: "Formatted at 2026-08-06 16:20:10 on dist-test-slave-t3q3"
server_key: "e33e7d4a1f5e2e692861cf7577b6dae0"
server_key_iv: "a1b2a56c423c2eaf703329eaea3923d6"
server_key_version: "encryptionkey@0"
I20260806 16:20:10.163841 32269 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 16:20:10.167016 32292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:10.168752 32269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:20:10.168928 32269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "16c771abcc6b41eea0cef489181e770c"
format_stamp: "Formatted at 2026-08-06 16:20:10 on dist-test-slave-t3q3"
server_key: "e33e7d4a1f5e2e692861cf7577b6dae0"
server_key_iv: "a1b2a56c423c2eaf703329eaea3923d6"
server_key_version: "encryptionkey@0"
I20260806 16:20:10.169094 32269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:10.183012 32269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:10.188226 32269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:10.188516 32269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:10.198361 32269 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:44261
I20260806 16:20:10.198741 32368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:44261 every 8 connection(s)
I20260806 16:20:10.199805 32269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:20:10.203578 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 32269
I20260806 16:20:10.203848 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:20:10.204290 32369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:10.204345 12246 external_mini_cluster.cc:1428] Setting key c914576035740443024be55f5d9cf0ca
I20260806 16:20:10.215085 32369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: Bootstrap starting.
Aug 06 16:20:10 dist-test-slave-t3q3 krb5kdc[32244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:10.218961 32369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:10.219899 32369 log.cc:824] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:10.222582 32369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: No bootstrap required, opened a new log
I20260806 16:20:10.223672 32372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:10.206374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52874 (local address 127.11.245.190:44261)
0806 16:20:10.206965 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:10.206986 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:10.207035 (+ 49us) server_negotiation.cc:407] Connection header received
0806 16:20:10.208564 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:10.208596 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:10.209090 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:10.209503 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:10.210720 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:10.211887 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:10.213413 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:10.214115 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:10.214274 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:10.217818 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:10.217825 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:10.217857 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:10.217899 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:10.220150 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:10.220716 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:10.220732 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:10.220739 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:10.220886 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:10.221212 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:10.221216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:10.221217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:10.221728 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:10.221983 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:10.222302 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 16:20:10.222711 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":192,"threads_started":1}
I20260806 16:20:10.227082 32369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } }
I20260806 16:20:10.227380 32369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:10.227442 32369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16c771abcc6b41eea0cef489181e770c, State: Initialized, Role: FOLLOWER
I20260806 16:20:10.228037 32369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } }
I20260806 16:20:10.228273 32369 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:20:10.228358 32369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:20:10.228490 32369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:10.229827 32369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } }
I20260806 16:20:10.230330 32369 leader_election.cc:302] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16c771abcc6b41eea0cef489181e770c; no voters:
I20260806 16:20:10.230798 32369 leader_election.cc:288] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:20:10.230985 32376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:20:10.231344 32376 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 1 LEADER]: Becoming Leader. State: Replica: 16c771abcc6b41eea0cef489181e770c, State: Running, Role: LEADER
I20260806 16:20:10.231890 32376 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } }
I20260806 16:20:10.233714 32369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:20:10.234193 32378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16c771abcc6b41eea0cef489181e770c. Latest consensus state: current_term: 1 leader_uuid: "16c771abcc6b41eea0cef489181e770c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } } }
I20260806 16:20:10.234797 32378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [sys.catalog]: This master's current role is: LEADER
I20260806 16:20:10.235239 32377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16c771abcc6b41eea0cef489181e770c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } } }
I20260806 16:20:10.235407 32377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [sys.catalog]: This master's current role is: LEADER
I20260806 16:20:10.235980 32381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:20:10.240692 32381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:20:10.248781 32381 catalog_manager.cc:1347] Generated new cluster ID: 3d526d088355494c8e352afef26824f3
I20260806 16:20:10.248912 32381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:20:10.274996 32381 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:20:10.276357 32381 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:20:10.284104 32381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: Generated new TSK 0
I20260806 16:20:10.285679 32381 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:20:10.332387 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:44261
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:20:10.484891 32404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:10.485301 32404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:10.485355 32404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:10.490635 32404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:10.490746 32404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:10.490819 32404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:20:10.497705 32404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:44261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:10.499197 32404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:10.500548 32404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:10.513916 32413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:10.513932 32416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:10.513949 32414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:10.514205 32404 server_base.cc:1034] running on GCE node
I20260806 16:20:10.514916 32404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:10.515837 32404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:10.517041 32404 hybrid_clock.cc:648] HybridClock initialized: now 1786033210517022 us; error 42 us; skew 500 ppm
Aug 06 16:20:10 dist-test-slave-t3q3 krb5kdc[32244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033210, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:10.520489 32404 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:20:10.521960 32404 webserver.cc:460] Webserver started at http://127.11.245.129:33425/ using document root <none> and password file <none>
I20260806 16:20:10.522854 32404 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:10.522997 32404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:10.523299 32404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:20:10.526084 32404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "892b4f4c5a9e45c79763b3ddf35a13e7"
format_stamp: "Formatted at 2026-08-06 16:20:10 on dist-test-slave-t3q3"
server_key: "fa3ecc6a04c8043816da2417053ca9e9"
server_key_iv: "27a87ac4c1e4249f19dd836ae64ddea0"
server_key_version: "encryptionkey@0"
I20260806 16:20:10.526784 32404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "892b4f4c5a9e45c79763b3ddf35a13e7"
format_stamp: "Formatted at 2026-08-06 16:20:10 on dist-test-slave-t3q3"
server_key: "fa3ecc6a04c8043816da2417053ca9e9"
server_key_iv: "27a87ac4c1e4249f19dd836ae64ddea0"
server_key_version: "encryptionkey@0"
I20260806 16:20:10.531729 32404 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 16:20:10.535356 32427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:10.537400 32404 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:20:10.537557 32404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "892b4f4c5a9e45c79763b3ddf35a13e7"
format_stamp: "Formatted at 2026-08-06 16:20:10 on dist-test-slave-t3q3"
server_key: "fa3ecc6a04c8043816da2417053ca9e9"
server_key_iv: "27a87ac4c1e4249f19dd836ae64ddea0"
server_key_version: "encryptionkey@0"
I20260806 16:20:10.537698 32404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:10.549088 32404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:10.553722 32404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:10.553953 32404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:10.554708 32404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:20:10.555787 32404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:20:10.555842 32404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:10.555892 32404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:20:10.555908 32404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:10.567122 32404 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:33525
I20260806 16:20:10.567165 32588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:33525 every 8 connection(s)
I20260806 16:20:10.568539 32404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:20:10.575160 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 32404
I20260806 16:20:10.575392 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:20:10.575771 12246 external_mini_cluster.cc:1428] Setting key d014e6402ee22e123cf00e3d2f1683c3
Aug 06 16:20:10 dist-test-slave-t3q3 krb5kdc[32244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033210, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:10.588883 32372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:10.571045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:37991 (local address 127.11.245.190:44261)
0806 16:20:10.571236 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:10.571241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:10.572429 (+ 1188us) server_negotiation.cc:407] Connection header received
0806 16:20:10.574250 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:10.574254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:10.574323 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:10.574424 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:10.576426 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:10.577159 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:10.578865 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:10.579321 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:10.579498 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:10.583592 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:10.583596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:10.583598 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:10.583633 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:10.585606 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:10.586552 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:10.586556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:10.586557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:10.586665 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:10.587207 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:10.587211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:10.587212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:10.587464 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:10.587555 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:10.588572 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0806 16:20:10.588718 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 16:20:10.590641 32594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:10.571566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:37991)
0806 16:20:10.572239 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:10.572293 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:10.573962 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:10.574699 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:10.574708 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:10.575384 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:10.576225 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:10.576247 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:10.577489 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:10.577496 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:10.578705 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:10.578718 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:10.579545 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:10.579548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:10.579762 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:10.580652 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:10.580708 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:10.583262 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:10.585777 (+ 2515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:10.585808 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:10.585841 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:10.586374 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:10.586785 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:10.586790 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:10.586794 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:10.587054 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:10.587617 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:10.587630 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:10.588172 (+ 542us) client_negotiation.cc:769] Sending connection context
0806 16:20:10.588508 (+ 336us) client_negotiation.cc:240] Negotiation successful
0806 16:20:10.589034 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":151,"threads_started":1}
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:20:10.592372 32590 heartbeater.cc:347] Connected to a master server at 127.11.245.190:44261
I20260806 16:20:10.592679 32590 heartbeater.cc:464] Registering TS with master...
I20260806 16:20:10.593379 32590 heartbeater.cc:511] Master 127.11.245.190:44261 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
I20260806 16:20:10.595687 32315 ts_manager.cc:194] Registered new tserver with Master: 892b4f4c5a9e45c79763b3ddf35a13e7 (127.11.245.129:33525)
I20260806 16:20:10.597536 32315 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:37991
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:20:10.616333 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:44261
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:20:10.773216 32598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:10.773686 32598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:10.773792 32598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:10.778231 32598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:10.778348 32598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:10.778468 32598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:20:10.784960 32598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:44261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:10.786733 32598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:10.788348 32598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:10.801255 32605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:10.801255 32608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:10.801312 32604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:10.801856 32598 server_base.cc:1034] running on GCE node
I20260806 16:20:10.802562 32598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:10.803558 32598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:10.804782 32598 hybrid_clock.cc:648] HybridClock initialized: now 1786033210804760 us; error 39 us; skew 500 ppm
Aug 06 16:20:10 dist-test-slave-t3q3 krb5kdc[32244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033210, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:10.809095 32598 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:20:10.810428 32598 webserver.cc:460] Webserver started at http://127.11.245.130:33249/ using document root <none> and password file <none>
I20260806 16:20:10.811251 32598 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:10.811359 32598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:10.811672 32598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:20:10.814563 32598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "ac47217cf3b946239b28ecbb1f383343"
format_stamp: "Formatted at 2026-08-06 16:20:10 on dist-test-slave-t3q3"
server_key: "08928eab00eb915fa60ae1502504d6c0"
server_key_iv: "a6bfd8dfa6781d1935d53ac465ffa13e"
server_key_version: "encryptionkey@0"
I20260806 16:20:10.815303 32598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "ac47217cf3b946239b28ecbb1f383343"
format_stamp: "Formatted at 2026-08-06 16:20:10 on dist-test-slave-t3q3"
server_key: "08928eab00eb915fa60ae1502504d6c0"
server_key_iv: "a6bfd8dfa6781d1935d53ac465ffa13e"
server_key_version: "encryptionkey@0"
I20260806 16:20:10.820978 32598 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:20:10.824510 32619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:10.826529 32598 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:20:10.826733 32598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "ac47217cf3b946239b28ecbb1f383343"
format_stamp: "Formatted at 2026-08-06 16:20:10 on dist-test-slave-t3q3"
server_key: "08928eab00eb915fa60ae1502504d6c0"
server_key_iv: "a6bfd8dfa6781d1935d53ac465ffa13e"
server_key_version: "encryptionkey@0"
I20260806 16:20:10.826908 32598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:10.860064 32598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:10.864322 32598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:10.864612 32598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:10.865489 32598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:20:10.866664 32598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:20:10.866743 32598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:10.866851 32598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:20:10.866896 32598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:10.878998 32598 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:33079
I20260806 16:20:10.879021 307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:33079 every 8 connection(s)
I20260806 16:20:10.880414 32598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:20:10.889045 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 32598
I20260806 16:20:10.889240 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:20:10.889881 12246 external_mini_cluster.cc:1428] Setting key 22b8a4812ac1bb758c20cb7a0f2efcea
Aug 06 16:20:10 dist-test-slave-t3q3 krb5kdc[32244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033210, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
I20260806 16:20:10.907248 32372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:10.883160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:36503 (local address 127.11.245.190:44261)
0806 16:20:10.883373 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:10.883378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:10.884640 (+ 1262us) server_negotiation.cc:407] Connection header received
0806 16:20:10.887498 (+ 2858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:10.887505 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:10.887649 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:10.887767 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:10.893097 (+ 5330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:10.894246 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:10.897122 (+ 2876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:10.897603 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:10.897715 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:10.902586 (+ 4871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:10.902591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:10.902593 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:10.902632 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:10.904542 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:10.905239 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:10.905242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:10.905244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:10.905350 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:10.905845 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:10.905848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:10.905849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:10.906105 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:10.906191 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:10.906943 (+ 752us) server_negotiation.cc:299] Negotiation successful
0806 16:20:10.907084 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
I20260806 16:20:10.908788 311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:10.883665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:36503)
0806 16:20:10.884406 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:10.884467 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:10.887030 (+ 2563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:10.888294 (+ 1264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:10.888307 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:10.891308 (+ 3001us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:10.892780 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:10.892825 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:10.894546 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:10.894551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:10.896882 (+ 2331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:10.896907 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:10.898079 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:10.898083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:10.898409 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:10.899405 (+ 996us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:10.899438 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:10.902250 (+ 2812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:10.904707 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:10.904716 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:10.904748 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:10.905101 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:10.905527 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:10.905530 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:10.905531 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:10.905724 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:10.906209 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:10.906217 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:10.906588 (+ 371us) client_negotiation.cc:769] Sending connection context
0806 16:20:10.906827 (+ 239us) client_negotiation.cc:240] Negotiation successful
0806 16:20:10.907257 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":165,"threads_started":1}
I20260806 16:20:10.912335 309 heartbeater.cc:347] Connected to a master server at 127.11.245.190:44261
I20260806 16:20:10.912714 309 heartbeater.cc:464] Registering TS with master...
I20260806 16:20:10.913565 309 heartbeater.cc:511] Master 127.11.245.190:44261 requested a full tablet report, sending...
I20260806 16:20:10.915212 32315 ts_manager.cc:194] Registered new tserver with Master: ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079)
I20260806 16:20:10.916337 32315 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:36503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:20:10.929898 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:44261
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:20:11.079051 314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:11.079449 314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:11.079502 314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:11.083930 314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:11.084045 314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:11.084139 314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:20:11.090278 314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:44261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:11.091801 314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:11.093155 314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:11.105757 322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:11.105844 324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:11.105870 314 server_base.cc:1034] running on GCE node
W20260806 16:20:11.105746 327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:11.106523 314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:11.107431 314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:11.108654 314 hybrid_clock.cc:648] HybridClock initialized: now 1786033211108600 us; error 69 us; skew 500 ppm
Aug 06 16:20:11 dist-test-slave-t3q3 krb5kdc[32244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033211, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:11.112531 314 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:20:11.113765 314 webserver.cc:460] Webserver started at http://127.11.245.131:42203/ using document root <none> and password file <none>
I20260806 16:20:11.114567 314 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:11.114665 314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:11.114957 314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:20:11.117995 314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "179757f75c874f76a99c7ce5d1369cdd"
format_stamp: "Formatted at 2026-08-06 16:20:11 on dist-test-slave-t3q3"
server_key: "091a32b073328a3d913582e2675c92c5"
server_key_iv: "c1ab866f22761cf526ac2f8354caff16"
server_key_version: "encryptionkey@0"
I20260806 16:20:11.118805 314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "179757f75c874f76a99c7ce5d1369cdd"
format_stamp: "Formatted at 2026-08-06 16:20:11 on dist-test-slave-t3q3"
server_key: "091a32b073328a3d913582e2675c92c5"
server_key_iv: "c1ab866f22761cf526ac2f8354caff16"
server_key_version: "encryptionkey@0"
I20260806 16:20:11.124343 314 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 16:20:11.127856 335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:11.129869 314 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:20:11.130110 314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "179757f75c874f76a99c7ce5d1369cdd"
format_stamp: "Formatted at 2026-08-06 16:20:11 on dist-test-slave-t3q3"
server_key: "091a32b073328a3d913582e2675c92c5"
server_key_iv: "c1ab866f22761cf526ac2f8354caff16"
server_key_version: "encryptionkey@0"
I20260806 16:20:11.130300 314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:11.157213 314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:11.161892 314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:11.162185 314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:11.163074 314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:20:11.164345 314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:20:11.164423 314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:11.164520 314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:20:11.164562 314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:11.176826 314 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:41759
I20260806 16:20:11.176931 493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:41759 every 8 connection(s)
I20260806 16:20:11.178104 314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:20:11.182471 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 314
I20260806 16:20:11.182631 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:20:11.183077 12246 external_mini_cluster.cc:1428] Setting key 2330189a5918a017bb1fa8c84d76b8ef
Aug 06 16:20:11 dist-test-slave-t3q3 krb5kdc[32244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033211, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:11.195822 32372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:11.180695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:59651 (local address 127.11.245.190:44261)
0806 16:20:11.180901 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:11.180907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:11.182068 (+ 1161us) server_negotiation.cc:407] Connection header received
0806 16:20:11.184276 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:11.184279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:11.184349 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:11.184440 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:11.186226 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.186826 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.188156 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.188400 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.188525 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:11.191534 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:11.191540 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:11.191543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:11.191580 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:11.193455 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:11.194085 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:11.194089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:11.194091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:11.194187 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:11.194536 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:11.194539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:11.194540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:11.194767 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:11.194855 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:11.195525 (+ 670us) server_negotiation.cc:299] Negotiation successful
0806 16:20:11.195669 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 16:20:11.197278 499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:11.181241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:59651)
0806 16:20:11.181878 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:11.181932 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:11.183950 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:11.184676 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:11.184688 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:11.185330 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:11.186058 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:11.186083 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.186970 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.186973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:11.187984 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:11.187995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.188582 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.188586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:11.188798 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:11.189539 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:11.189558 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:11.191247 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:11.193592 (+ 2345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:11.193598 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:11.193614 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:11.193954 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:11.194285 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:11.194287 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:11.194288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:11.194447 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:11.194900 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:11.194907 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:11.195234 (+ 327us) client_negotiation.cc:769] Sending connection context
0806 16:20:11.195447 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 16:20:11.195830 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":156,"threads_started":1}
I20260806 16:20:11.198810 497 heartbeater.cc:347] Connected to a master server at 127.11.245.190:44261
I20260806 16:20:11.199141 497 heartbeater.cc:464] Registering TS with master...
I20260806 16:20:11.199910 497 heartbeater.cc:511] Master 127.11.245.190:44261 requested a full tablet report, sending...
I20260806 16:20:11.201422 32312 ts_manager.cc:194] Registered new tserver with Master: 179757f75c874f76a99c7ce5d1369cdd (127.11.245.131:41759)
I20260806 16:20:11.202193 32312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:59651
I20260806 16:20:11.211485 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:20:11.222662 32372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:11.214266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52880 (local address 127.11.245.190:44261)
0806 16:20:11.214444 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:11.214449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:11.214549 (+ 100us) server_negotiation.cc:407] Connection header received
0806 16:20:11.214676 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:11.214678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:11.214742 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:11.214849 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:11.215675 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.216385 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.217329 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.217604 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.217706 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:11.219361 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:11.219365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:11.219367 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:11.219399 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:11.220902 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:11.221387 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:11.221390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:11.221391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:11.221470 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:11.221727 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:11.221729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:11.221731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:11.221990 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:11.222109 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:11.222381 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 16:20:11.222494 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 16:20:11.226755 32312 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:20:11.230773 32312 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:20:11.250941 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:11.244190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47352 (local address 127.11.245.131:41759)
0806 16:20:11.245045 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:11.245054 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:11.245074 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:20:11.245149 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:11.245153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:11.245554 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:11.245791 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:11.246073 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.247416 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.249020 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.249964 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.250121 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:11.250342 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:11.250578 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 16:20:11.250712 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":190,"threads_started":1}
I20260806 16:20:11.250942 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:11.243254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:33079 (local address 127.0.0.1:34130)
0806 16:20:11.244374 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:11.244388 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:11.244482 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:11.245383 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:11.245391 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:11.245417 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:11.245629 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:11.245647 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.247249 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.247254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:11.248872 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:11.248884 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.250286 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.250290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:11.250392 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:11.250416 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 16:20:11.250492 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 16:20:11.250720 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1036,"mutex_wait_us":154,"thread_start_us":160,"threads_started":1}
I20260806 16:20:11.250980 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:11.243134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:41759 (local address 127.0.0.1:47352)
0806 16:20:11.243809 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:11.243885 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:11.244084 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:11.245761 (+ 1677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:11.245765 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:11.245785 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:11.245940 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:11.245948 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.247693 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.247697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:11.248755 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:11.248766 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.250102 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.250105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:11.250253 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:11.250298 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 16:20:11.250531 (+ 233us) client_negotiation.cc:240] Negotiation successful
0806 16:20:11.250720 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":72,"thread_start_us":112,"threads_started":1}
I20260806 16:20:11.251530 515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:11.243587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:33525 (local address 127.0.0.1:39056)
0806 16:20:11.244697 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:11.244709 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:11.244796 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:11.245771 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:11.245773 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:11.245804 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:11.245948 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:11.245967 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.247705 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.247709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:11.249021 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:11.249030 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.250505 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.250509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:11.250619 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:11.250645 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 16:20:11.250711 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 16:20:11.250778 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"mutex_wait_us":246,"spinlock_wait_cycles":1792,"thread_start_us":133,"threads_started":1}
I20260806 16:20:11.251593 516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:11.244227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39056 (local address 127.11.245.129:33525)
0806 16:20:11.244790 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:11.244797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:11.244826 (+ 29us) server_negotiation.cc:407] Connection header received
0806 16:20:11.244924 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:11.244930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:11.245181 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:11.245372 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:11.246120 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.247419 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.249165 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.250125 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.250286 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:11.250436 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:11.250918 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 16:20:11.251068 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":139,"threads_started":1}
I20260806 16:20:11.251731 518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:11.244252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34130 (local address 127.11.245.130:33079)
0806 16:20:11.244775 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:11.244797 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:11.244826 (+ 29us) server_negotiation.cc:407] Connection header received
0806 16:20:11.244924 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:11.244930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:11.245182 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:11.245367 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:11.246211 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.247045 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.249031 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.250125 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.251220 (+ 1095us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:11.251371 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:11.251425 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 16:20:11.251543 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":146,"threads_started":1}
I20260806 16:20:11.255936 32494 tablet_service.cc:1467] Processing CreateTablet for tablet 5b5fadb0e0d84ec9afb3d3226c5e0581 (DEFAULT_TABLE table=test-table [id=c64af47a57824e8abbc216c00c45cd71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:20:11.255936 408 tablet_service.cc:1467] Processing CreateTablet for tablet 5b5fadb0e0d84ec9afb3d3226c5e0581 (DEFAULT_TABLE table=test-table [id=c64af47a57824e8abbc216c00c45cd71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:20:11.255936 32683 tablet_service.cc:1467] Processing CreateTablet for tablet 5b5fadb0e0d84ec9afb3d3226c5e0581 (DEFAULT_TABLE table=test-table [id=c64af47a57824e8abbc216c00c45cd71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:20:11.257520 32683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b5fadb0e0d84ec9afb3d3226c5e0581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:11.257534 408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b5fadb0e0d84ec9afb3d3226c5e0581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:11.257520 32494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b5fadb0e0d84ec9afb3d3226c5e0581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:11.266564 520 tablet_bootstrap.cc:492] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343: Bootstrap starting.
I20260806 16:20:11.267143 522 tablet_bootstrap.cc:492] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: Bootstrap starting.
I20260806 16:20:11.270119 522 tablet_bootstrap.cc:654] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:11.270705 520 tablet_bootstrap.cc:654] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:11.270738 521 tablet_bootstrap.cc:492] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd: Bootstrap starting.
I20260806 16:20:11.271279 522 log.cc:824] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:11.271673 520 log.cc:824] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:11.273510 521 tablet_bootstrap.cc:654] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:11.274978 522 tablet_bootstrap.cc:492] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: No bootstrap required, opened a new log
I20260806 16:20:11.275094 520 tablet_bootstrap.cc:492] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343: No bootstrap required, opened a new log
I20260806 16:20:11.275051 521 log.cc:824] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:11.275280 522 ts_tablet_manager.cc:1397] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 16:20:11.275310 520 ts_tablet_manager.cc:1397] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.002s
I20260806 16:20:11.277602 521 tablet_bootstrap.cc:492] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd: No bootstrap required, opened a new log
I20260806 16:20:11.277882 521 ts_tablet_manager.cc:1397] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 16:20:11.279834 520 raft_consensus.cc:348] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:11.279829 522 raft_consensus.cc:348] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:11.280224 520 raft_consensus.cc:374] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:11.280244 522 raft_consensus.cc:374] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:11.280294 520 raft_consensus.cc:729] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac47217cf3b946239b28ecbb1f383343, State: Initialized, Role: FOLLOWER
I20260806 16:20:11.280305 522 raft_consensus.cc:729] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892b4f4c5a9e45c79763b3ddf35a13e7, State: Initialized, Role: FOLLOWER
I20260806 16:20:11.280916 520 consensus_queue.cc:260] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:11.281445 522 consensus_queue.cc:260] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:11.282291 520 ts_tablet_manager.cc:1428] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343: Time spent starting tablet: real 0.007s user 0.006s sys 0.002s
I20260806 16:20:11.282701 32590 heartbeater.cc:503] Master 127.11.245.190:44261 was elected leader, sending a full tablet report...
I20260806 16:20:11.282723 309 heartbeater.cc:503] Master 127.11.245.190:44261 was elected leader, sending a full tablet report...
I20260806 16:20:11.282297 521 raft_consensus.cc:348] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:11.283871 521 raft_consensus.cc:374] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:11.283967 521 raft_consensus.cc:729] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 179757f75c874f76a99c7ce5d1369cdd, State: Initialized, Role: FOLLOWER
I20260806 16:20:11.284008 522 ts_tablet_manager.cc:1428] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260806 16:20:11.284587 521 consensus_queue.cc:260] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:11.285429 497 heartbeater.cc:503] Master 127.11.245.190:44261 was elected leader, sending a full tablet report...
I20260806 16:20:11.285717 521 ts_tablet_manager.cc:1428] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
W20260806 16:20:11.320537 32591 tablet.cc:2369] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:20:11.362548 528 raft_consensus.cc:482] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:20:11.362882 528 raft_consensus.cc:504] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:11.364763 528 leader_election.cc:288] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079), 892b4f4c5a9e45c79763b3ddf35a13e7 (127.11.245.129:33525)
I20260806 16:20:11.369539 499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:11.365179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:33079 (local address 127.11.245.131:37469)
0806 16:20:11.365480 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:11.365500 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:11.365645 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:11.365985 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:11.365988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:11.366007 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:11.366162 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:11.366167 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.367008 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.367012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:11.368184 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:11.368196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.368985 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.368989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:11.369113 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:11.369155 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 16:20:11.369245 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 16:20:11.369339 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179}
I20260806 16:20:11.369537 518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:11.365308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:37469 (local address 127.11.245.130:33079)
0806 16:20:11.365504 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:11.365511 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:11.365625 (+ 114us) server_negotiation.cc:407] Connection header received
0806 16:20:11.365801 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:11.365804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:11.365865 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:11.365954 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:11.366294 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.366877 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.368337 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.368834 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.368937 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:11.369000 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:11.369274 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 16:20:11.369371 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 16:20:11.369760 529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:11.365179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:33525 (local address 127.11.245.131:56781)
0806 16:20:11.365759 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:11.365771 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:11.365930 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:11.366234 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:11.366237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:11.366252 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:11.366393 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:11.366399 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.367290 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.367296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:11.368375 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:11.368386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.369282 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.369286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:11.369405 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:11.369431 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 16:20:11.369521 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 16:20:11.369607 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":65,"thread_start_us":184,"threads_started":1}
I20260806 16:20:11.369823 516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:11.365466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:56781 (local address 127.11.245.129:33525)
0806 16:20:11.365616 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:11.365620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:11.365886 (+ 266us) server_negotiation.cc:407] Connection header received
0806 16:20:11.366077 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:11.366080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:11.366131 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:11.366225 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:11.366519 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.367171 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.368537 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.369114 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.369264 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:11.369317 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:11.369566 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 16:20:11.369669 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 16:20:11.370599 32709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b5fadb0e0d84ec9afb3d3226c5e0581" candidate_uuid: "179757f75c874f76a99c7ce5d1369cdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac47217cf3b946239b28ecbb1f383343" is_pre_election: true
I20260806 16:20:11.370759 32523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b5fadb0e0d84ec9afb3d3226c5e0581" candidate_uuid: "179757f75c874f76a99c7ce5d1369cdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" is_pre_election: true
I20260806 16:20:11.371106 32523 raft_consensus.cc:2454] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 179757f75c874f76a99c7ce5d1369cdd in term 0.
I20260806 16:20:11.371106 32709 raft_consensus.cc:2454] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 179757f75c874f76a99c7ce5d1369cdd in term 0.
I20260806 16:20:11.371727 337 leader_election.cc:302] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 179757f75c874f76a99c7ce5d1369cdd, 892b4f4c5a9e45c79763b3ddf35a13e7, ac47217cf3b946239b28ecbb1f383343; no voters:
I20260806 16:20:11.372040 528 raft_consensus.cc:2781] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:20:11.372164 528 raft_consensus.cc:482] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:20:11.372219 528 raft_consensus.cc:3037] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:11.373665 528 raft_consensus.cc:504] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:11.374074 528 leader_election.cc:288] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [CANDIDATE]: Term 1 election: Requested vote from peers ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079), 892b4f4c5a9e45c79763b3ddf35a13e7 (127.11.245.129:33525)
I20260806 16:20:11.374467 32709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b5fadb0e0d84ec9afb3d3226c5e0581" candidate_uuid: "179757f75c874f76a99c7ce5d1369cdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac47217cf3b946239b28ecbb1f383343"
I20260806 16:20:11.374557 32523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b5fadb0e0d84ec9afb3d3226c5e0581" candidate_uuid: "179757f75c874f76a99c7ce5d1369cdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7"
I20260806 16:20:11.374624 32709 raft_consensus.cc:3037] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:11.374701 32523 raft_consensus.cc:3037] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:11.376296 32523 raft_consensus.cc:2454] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 179757f75c874f76a99c7ce5d1369cdd in term 1.
I20260806 16:20:11.376302 32709 raft_consensus.cc:2454] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 179757f75c874f76a99c7ce5d1369cdd in term 1.
I20260806 16:20:11.376679 338 leader_election.cc:302] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 179757f75c874f76a99c7ce5d1369cdd, ac47217cf3b946239b28ecbb1f383343; no voters:
I20260806 16:20:11.376919 528 raft_consensus.cc:2781] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:20:11.377214 528 raft_consensus.cc:686] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [term 1 LEADER]: Becoming Leader. State: Replica: 179757f75c874f76a99c7ce5d1369cdd, State: Running, Role: LEADER
I20260806 16:20:11.377724 528 consensus_queue.cc:237] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
W20260806 16:20:11.382187 310 tablet.cc:2369] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:20:11.382614 32315 catalog_manager.cc:5495] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd reported cstate change: term changed from 0 to 1, leader changed from <none> to 179757f75c874f76a99c7ce5d1369cdd (127.11.245.131). New cstate: current_term: 1 leader_uuid: "179757f75c874f76a99c7ce5d1369cdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } health_report { overall_health: UNKNOWN } } }
I20260806 16:20:11.409307 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:11.405164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47358 (local address 127.11.245.131:41759)
0806 16:20:11.405352 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:11.405356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:11.405370 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:20:11.405484 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:11.405486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:11.405557 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:11.405666 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:20:11.405945 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.406523 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.407395 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:11.407640 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:11.407727 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:11.408174 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:20:11.408847 (+ 673us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:20:11.408975 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:11.409034 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 16:20:11.409110 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 16:20:11.417659 32709 raft_consensus.cc:1261] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343 [term 1 FOLLOWER]: Refusing update from remote peer 179757f75c874f76a99c7ce5d1369cdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:20:11.417660 32523 raft_consensus.cc:1261] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Refusing update from remote peer 179757f75c874f76a99c7ce5d1369cdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:20:11.418685 530 consensus_queue.cc:1035] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:20:11.418962 528 consensus_queue.cc:1035] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:20:11.430320 536 mvcc.cc:205] Tried to move back new op lower bound from 7315592033956544512 to 7315592033806610432. Current Snapshot: MvccSnapshot[applied={T|T < 7315592033956544512}]
I20260806 16:20:11.445461 32315 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:20:11.445770 32315 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:20:11.450060 32315 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: Sending DeleteTablet for 3 replicas of tablet 5b5fadb0e0d84ec9afb3d3226c5e0581
I20260806 16:20:11.451424 408 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b5fadb0e0d84ec9afb3d3226c5e0581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:20:11 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:47352
I20260806 16:20:11.451728 32683 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b5fadb0e0d84ec9afb3d3226c5e0581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:20:11 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:34130
I20260806 16:20:11.452031 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 32404
I20260806 16:20:11.458214 552 tablet_replica.cc:330] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd: stopping tablet replica
I20260806 16:20:11.458873 552 raft_consensus.cc:2229] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:20:11.459652 552 raft_consensus.cc:2258] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:20:11.465860 552 ts_tablet_manager.cc:1905] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:20:11.472568 553 tablet_replica.cc:330] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343: stopping tablet replica
I20260806 16:20:11.472944 552 ts_tablet_manager.cc:1918] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:20:11.473326 552 log.cc:1196] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/5b5fadb0e0d84ec9afb3d3226c5e0581
I20260806 16:20:11.473560 553 raft_consensus.cc:2229] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:20:11.474571 552 ts_tablet_manager.cc:1939] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 179757f75c874f76a99c7ce5d1369cdd: Deleting consensus metadata
I20260806 16:20:11.474725 553 raft_consensus.cc:2258] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:20:11.476958 32293 catalog_manager.cc:4841] TS 179757f75c874f76a99c7ce5d1369cdd (127.11.245.131:41759): tablet 5b5fadb0e0d84ec9afb3d3226c5e0581 (table test-table [id=c64af47a57824e8abbc216c00c45cd71]) successfully deleted
I20260806 16:20:11.477319 553 ts_tablet_manager.cc:1905] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:20:11.481045 553 ts_tablet_manager.cc:1918] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:20:11.481235 553 log.cc:1196] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/5b5fadb0e0d84ec9afb3d3226c5e0581
I20260806 16:20:11.481592 553 ts_tablet_manager.cc:1939] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P ac47217cf3b946239b28ecbb1f383343: Deleting consensus metadata
W20260806 16:20:11.482389 32295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:33525 (error 108)
I20260806 16:20:11.482499 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 32598
I20260806 16:20:11.492244 515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:11.491060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:33525 (local address 127.0.0.1:39068)
0806 16:20:11.491911 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:11.492042 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:33525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":683}
W20260806 16:20:11.493753 32295 catalog_manager.cc:4584] TS 892b4f4c5a9e45c79763b3ddf35a13e7 (127.11.245.129:33525): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b5fadb0e0d84ec9afb3d3226c5e0581: Network error: Client connection negotiation failed: client connection to 127.11.245.129:33525: connect: Connection refused (error 111)
I20260806 16:20:11.505705 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 314
I20260806 16:20:11.505857 515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:11.505450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:33079 (local address 127.0.0.1:34138)
0806 16:20:11.505655 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:11.505725 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.130:33079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 16:20:11.525462 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 32269
I20260806 16:20:11.544724 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:44261
--webserver_interface=127.11.245.190
--webserver_port=40297
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:44261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:20:11.694895 556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:11.695298 556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:11.695348 556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:11.699887 556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:20:11.699996 556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:11.700018 556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:11.700037 556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:20:11.700084 556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:20:11.707091 556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:44261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:44261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=40297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:11.709182 556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:11.710690 556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:11.730903 571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:11.730906 573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:11.730903 567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:11.731549 556 server_base.cc:1034] running on GCE node
I20260806 16:20:11.732645 556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:11.734381 556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:11.735615 556 hybrid_clock.cc:648] HybridClock initialized: now 1786033211735583 us; error 64 us; skew 500 ppm
Aug 06 16:20:11 dist-test-slave-t3q3 krb5kdc[32244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033211, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:11.739486 556 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:20:11.740917 556 webserver.cc:460] Webserver started at http://127.11.245.190:40297/ using document root <none> and password file <none>
I20260806 16:20:11.741776 556 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:11.741918 556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:11.747748 556 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:20:11.751284 580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:11.753240 556 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:20:11.753532 556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "16c771abcc6b41eea0cef489181e770c"
format_stamp: "Formatted at 2026-08-06 16:20:10 on dist-test-slave-t3q3"
server_key: "e33e7d4a1f5e2e692861cf7577b6dae0"
server_key_iv: "a1b2a56c423c2eaf703329eaea3923d6"
server_key_version: "encryptionkey@0"
I20260806 16:20:11.754724 556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:11.775081 556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:11.779946 556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:11.780279 556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:11.790784 556 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:44261
I20260806 16:20:11.791134 654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:44261 every 8 connection(s)
I20260806 16:20:11.792783 556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:20:11.797036 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 556
I20260806 16:20:11.797706 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:33525
--local_ip_for_outbound_sockets=127.11.245.129
--tserver_master_addrs=127.11.245.190:44261
--webserver_port=33425
--webserver_interface=127.11.245.129
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:20:11.799716 656 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:20:11.802213 656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: Bootstrap starting.
I20260806 16:20:11.830415 656 log.cc:824] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:11.841879 656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:20:11.842423 656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: Bootstrap complete.
I20260806 16:20:11.847571 656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } }
I20260806 16:20:11.848049 656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16c771abcc6b41eea0cef489181e770c, State: Initialized, Role: FOLLOWER
I20260806 16:20:11.848801 656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } }
I20260806 16:20:11.849066 656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:20:11.849172 656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:20:11.849293 656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 1 FOLLOWER]: Advancing to term 2
I20260806 16:20:11.851658 656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } }
I20260806 16:20:11.852141 656 leader_election.cc:302] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16c771abcc6b41eea0cef489181e770c; no voters:
I20260806 16:20:11.852605 656 leader_election.cc:288] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 16:20:11.852730 661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 2 FOLLOWER]: Leader election won for term 2
I20260806 16:20:11.852979 661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 2 LEADER]: Becoming Leader. State: Replica: 16c771abcc6b41eea0cef489181e770c, State: Running, Role: LEADER
I20260806 16:20:11.853411 661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } }
I20260806 16:20:11.854257 656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:20:11.855234 663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16c771abcc6b41eea0cef489181e770c. Latest consensus state: current_term: 2 leader_uuid: "16c771abcc6b41eea0cef489181e770c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } } }
I20260806 16:20:11.855278 662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "16c771abcc6b41eea0cef489181e770c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } } }
I20260806 16:20:11.855392 663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [sys.catalog]: This master's current role is: LEADER
I20260806 16:20:11.855404 662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [sys.catalog]: This master's current role is: LEADER
I20260806 16:20:11.855816 671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:20:11.862236 671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:20:11.863128 671 catalog_manager.cc:1259] Loaded cluster ID: 3d526d088355494c8e352afef26824f3
I20260806 16:20:11.863241 671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:20:11.864984 671 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 16:20:11.869971 671 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 16:20:11.877939 671 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:20:11.879079 671 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: Loaded TSK: 0
I20260806 16:20:11.880877 671 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 16:20:11.968339 659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:11.968713 659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:11.968765 659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:11.975082 659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:11.975214 659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:11.975294 659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:20:11.983866 659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:33525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=33425
--tserver_master_addrs=127.11.245.190:44261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:11.985646 659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:11.987146 659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:12.000730 692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:12.000881 694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:12.000722 691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:12.001693 659 server_base.cc:1034] running on GCE node
I20260806 16:20:12.002386 659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:12.003423 659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:12.004670 659 hybrid_clock.cc:648] HybridClock initialized: now 1786033212004621 us; error 59 us; skew 500 ppm
Aug 06 16:20:12 dist-test-slave-t3q3 krb5kdc[32244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033212, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:12.009173 659 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:20:12.010552 659 webserver.cc:460] Webserver started at http://127.11.245.129:33425/ using document root <none> and password file <none>
I20260806 16:20:12.011323 659 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:12.011466 659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:12.017107 659 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:20:12.020838 702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:12.022822 659 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:20:12.023018 659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "892b4f4c5a9e45c79763b3ddf35a13e7"
format_stamp: "Formatted at 2026-08-06 16:20:10 on dist-test-slave-t3q3"
server_key: "fa3ecc6a04c8043816da2417053ca9e9"
server_key_iv: "27a87ac4c1e4249f19dd836ae64ddea0"
server_key_version: "encryptionkey@0"
I20260806 16:20:12.023870 659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:12.042994 659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:12.047335 659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:12.047655 659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:12.048534 659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:20:12.050102 711 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 16:20:12.056198 659 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 16:20:12.056346 659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260806 16:20:12.056484 659 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 16:20:12.058923 659 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 16:20:12.059008 659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260806 16:20:12.059407 711 tablet_bootstrap.cc:492] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: Bootstrap starting.
I20260806 16:20:12.072412 659 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:33525
I20260806 16:20:12.072484 870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:33525 every 8 connection(s)
I20260806 16:20:12.073727 659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:20:12.083789 711 log.cc:824] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:12.084386 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 659
I20260806 16:20:12.084996 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:33079
--local_ip_for_outbound_sockets=127.11.245.130
--tserver_master_addrs=127.11.245.190:44261
--webserver_port=33249
--webserver_interface=127.11.245.130
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:20:12 dist-test-slave-t3q3 krb5kdc[32244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033212, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:12.095988 711 tablet_bootstrap.cc:492] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:20:12.096534 711 tablet_bootstrap.cc:492] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: Bootstrap complete.
I20260806 16:20:12.098587 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.077106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:55843 (local address 127.11.245.190:44261)
0806 16:20:12.077681 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:12.077706 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:12.078067 (+ 361us) server_negotiation.cc:407] Connection header received
0806 16:20:12.079532 (+ 1465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:12.079549 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:12.080114 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:12.080713 (+ 599us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:12.082090 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.083799 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.086047 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.086818 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.087050 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.090567 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:12.090577 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:12.090597 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:12.090643 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:12.093256 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:12.094207 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:12.094215 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:12.094221 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:12.094420 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:12.094933 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:12.094937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:12.094939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:12.095539 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:12.095706 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:12.096919 (+ 1213us) server_negotiation.cc:299] Negotiation successful
0806 16:20:12.097318 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":182,"threads_started":1}
I20260806 16:20:12.098590 711 ts_tablet_manager.cc:1397] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: Time spent bootstrapping tablet: real 0.040s user 0.032s sys 0.006s
I20260806 16:20:12.098604 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.077025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:55843)
0806 16:20:12.077831 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:12.077897 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:12.078940 (+ 1043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:12.080750 (+ 1810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:12.080760 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:12.081214 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:12.081843 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:12.081863 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.084193 (+ 2330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.084201 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:12.085838 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:12.085862 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.087018 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.087025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:12.087314 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.088318 (+ 1004us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:12.088341 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:12.090254 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:12.093438 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:12.093449 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:12.093470 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:12.094034 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:12.094555 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:12.094561 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:12.094566 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:12.094803 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:12.095880 (+ 1077us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:12.095891 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:12.096342 (+ 451us) client_negotiation.cc:769] Sending connection context
0806 16:20:12.096751 (+ 409us) client_negotiation.cc:240] Negotiation successful
0806 16:20:12.097313 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":196,"threads_started":1}
I20260806 16:20:12.102254 871 heartbeater.cc:347] Connected to a master server at 127.11.245.190:44261
I20260806 16:20:12.101965 711 raft_consensus.cc:348] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:12.102553 871 heartbeater.cc:464] Registering TS with master...
I20260806 16:20:12.102583 711 raft_consensus.cc:729] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892b4f4c5a9e45c79763b3ddf35a13e7, State: Initialized, Role: FOLLOWER
I20260806 16:20:12.103165 711 consensus_queue.cc:260] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:12.103313 871 heartbeater.cc:511] Master 127.11.245.190:44261 requested a full tablet report, sending...
I20260806 16:20:12.104769 711 ts_tablet_manager.cc:1428] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 16:20:12.107820 603 ts_manager.cc:194] Registered new tserver with Master: 892b4f4c5a9e45c79763b3ddf35a13e7 (127.11.245.129:33525)
I20260806 16:20:12.113255 603 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:55843
I20260806 16:20:12.117266 871 heartbeater.cc:503] Master 127.11.245.190:44261 was elected leader, sending a full tablet report...
Aug 06 16:20:12 dist-test-slave-t3q3 krb5kdc[32244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033211, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for kudu/127.11.245.129@KRBTEST.COM
I20260806 16:20:12.125643 879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.112493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:33525 (local address 127.0.0.1:39084)
0806 16:20:12.113397 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:12.113468 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:12.113682 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:12.114355 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:12.114361 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:12.114962 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:12.115102 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:12.115111 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.116270 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.116275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:12.116946 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:12.116956 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.117673 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.117677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:12.117819 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.118549 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:12.118562 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:12.120795 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:12.123104 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:12.123115 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:12.123123 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:12.123414 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:12.123777 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:12.123781 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:12.123783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:12.123887 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:12.124533 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:12.124541 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:12.124705 (+ 164us) client_negotiation.cc:769] Sending connection context
0806 16:20:12.125016 (+ 311us) client_negotiation.cc:240] Negotiation successful
0806 16:20:12.125345 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":146,"threads_started":1}
I20260806 16:20:12.125643 881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.112570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39084 (local address 127.11.245.129:33525)
0806 16:20:12.113738 (+ 1168us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:12.113755 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:12.113814 (+ 59us) server_negotiation.cc:407] Connection header received
0806 16:20:12.113913 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:12.113919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:12.114160 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:12.114313 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:12.115269 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.116156 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.117085 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.117530 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.117660 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.120980 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:12.120993 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:12.121014 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:12.121044 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:12.122976 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:12.123564 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:12.123571 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:12.123572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:12.123670 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:12.123990 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:12.123995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:12.123997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:12.124400 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:12.124568 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:12.125127 (+ 559us) server_negotiation.cc:299] Negotiation successful
0806 16:20:12.125366 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":1055,"thread_start_us":160,"threads_started":1}
I20260806 16:20:12.127599 783 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b5fadb0e0d84ec9afb3d3226c5e0581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:20:11 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:39084
I20260806 16:20:12.128769 882 tablet_replica.cc:330] stopping tablet replica
I20260806 16:20:12.129171 882 raft_consensus.cc:2229] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:20:12.129587 882 raft_consensus.cc:2258] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:20:12.130893 882 ts_tablet_manager.cc:1905] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:20:12.135653 882 ts_tablet_manager.cc:1918] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:20:12.135757 882 log.cc:1196] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal/wals/5b5fadb0e0d84ec9afb3d3226c5e0581
I20260806 16:20:12.136173 882 ts_tablet_manager.cc:1939] T 5b5fadb0e0d84ec9afb3d3226c5e0581 P 892b4f4c5a9e45c79763b3ddf35a13e7: Deleting consensus metadata
I20260806 16:20:12.138415 582 catalog_manager.cc:4841] TS 892b4f4c5a9e45c79763b3ddf35a13e7 (127.11.245.129:33525): tablet 5b5fadb0e0d84ec9afb3d3226c5e0581 (table test-table [id=c64af47a57824e8abbc216c00c45cd71]) successfully deleted
W20260806 16:20:12.246362 877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:12.246730 877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:12.246783 877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:12.252157 877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:12.252272 877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:12.252354 877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:20:12.259516 877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:33079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=33249
--tserver_master_addrs=127.11.245.190:44261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:12.261176 877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:12.262601 877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:12.275482 892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:12.275432 887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:12.275530 888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:12.275965 877 server_base.cc:1034] running on GCE node
I20260806 16:20:12.276604 877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:12.277500 877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:12.278729 877 hybrid_clock.cc:648] HybridClock initialized: now 1786033212278711 us; error 33 us; skew 500 ppm
Aug 06 16:20:12 dist-test-slave-t3q3 krb5kdc[32244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033212, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:12.282859 877 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:20:12.284384 877 webserver.cc:460] Webserver started at http://127.11.245.130:33249/ using document root <none> and password file <none>
I20260806 16:20:12.285517 877 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:12.285619 877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:12.291093 877 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 16:20:12.294055 901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:12.295852 877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:20:12.296084 877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "ac47217cf3b946239b28ecbb1f383343"
format_stamp: "Formatted at 2026-08-06 16:20:10 on dist-test-slave-t3q3"
server_key: "08928eab00eb915fa60ae1502504d6c0"
server_key_iv: "a6bfd8dfa6781d1935d53ac465ffa13e"
server_key_version: "encryptionkey@0"
I20260806 16:20:12.296983 877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:12.309648 877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:12.313611 877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:12.313853 877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:12.314622 877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:20:12.315675 877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:20:12.315727 877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:12.315774 877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:20:12.315830 877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:12.326566 877 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:33079
I20260806 16:20:12.326587 1075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:33079 every 8 connection(s)
I20260806 16:20:12.328286 877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:20:12.337800 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 877
I20260806 16:20:12.338377 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:41759
--local_ip_for_outbound_sockets=127.11.245.131
--tserver_master_addrs=127.11.245.190:44261
--webserver_port=42203
--webserver_interface=127.11.245.131
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:20:12 dist-test-slave-t3q3 krb5kdc[32244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033212, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:12.346459 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.330695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:46821 (local address 127.11.245.190:44261)
0806 16:20:12.330880 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:12.330890 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:12.331882 (+ 992us) server_negotiation.cc:407] Connection header received
0806 16:20:12.333572 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:12.333575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:12.333642 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:12.333726 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:12.335149 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.335745 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.337255 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.337646 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.337777 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.341474 (+ 3697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:12.341478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:12.341480 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:12.341514 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:12.343330 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:12.344165 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:12.344169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:12.344170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:12.344255 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:12.344746 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:12.344749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:12.344750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:12.344996 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:12.345078 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:12.346109 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0806 16:20:12.346275 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:20:12.348203 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.331105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:46821)
0806 16:20:12.331676 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:12.331725 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:12.333287 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:12.333933 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:12.333941 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:12.334402 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:12.334994 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:12.335009 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.335893 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.335897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:12.337106 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:12.337119 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.337850 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.337854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:12.338093 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.338898 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:12.338917 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:12.341207 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:12.343477 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:12.343489 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:12.343515 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:12.343984 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:12.344360 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:12.344367 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:12.344372 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:12.344612 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:12.345131 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:12.345143 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:12.345628 (+ 485us) client_negotiation.cc:769] Sending connection context
0806 16:20:12.345925 (+ 297us) client_negotiation.cc:240] Negotiation successful
0806 16:20:12.346477 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":135,"threads_started":1}
I20260806 16:20:12.349752 1076 heartbeater.cc:347] Connected to a master server at 127.11.245.190:44261
I20260806 16:20:12.350091 1076 heartbeater.cc:464] Registering TS with master...
I20260806 16:20:12.350785 1076 heartbeater.cc:511] Master 127.11.245.190:44261 requested a full tablet report, sending...
I20260806 16:20:12.352219 603 ts_manager.cc:194] Registered new tserver with Master: ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079)
I20260806 16:20:12.352931 603 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:46821
W20260806 16:20:12.495885 1079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:12.496385 1079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:12.496438 1079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:12.500689 1079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:12.500769 1079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:12.500844 1079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:20:12.507851 1079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:41759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=42203
--tserver_master_addrs=127.11.245.190:44261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:12.509691 1079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:12.511164 1079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:12.523660 1087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:12.523885 1088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:12.523669 1090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:12.523958 1079 server_base.cc:1034] running on GCE node
I20260806 16:20:12.524590 1079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:12.525434 1079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:12.526638 1079 hybrid_clock.cc:648] HybridClock initialized: now 1786033212526619 us; error 33 us; skew 500 ppm
Aug 06 16:20:12 dist-test-slave-t3q3 krb5kdc[32244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033212, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:12.530396 1079 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:20:12.531577 1079 webserver.cc:460] Webserver started at http://127.11.245.131:42203/ using document root <none> and password file <none>
I20260806 16:20:12.532397 1079 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:12.532524 1079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:12.537631 1079 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:20:12.540550 1101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:12.542383 1079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 16:20:12.542573 1079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "179757f75c874f76a99c7ce5d1369cdd"
format_stamp: "Formatted at 2026-08-06 16:20:11 on dist-test-slave-t3q3"
server_key: "091a32b073328a3d913582e2675c92c5"
server_key_iv: "c1ab866f22761cf526ac2f8354caff16"
server_key_version: "encryptionkey@0"
I20260806 16:20:12.543355 1079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:12.561213 1079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:12.565420 1079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:12.565681 1079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:12.566504 1079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:20:12.567603 1079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:20:12.567682 1079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:12.567762 1079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:20:12.567817 1079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:12.579696 1079 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:41759
I20260806 16:20:12.579728 1264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:41759 every 8 connection(s)
I20260806 16:20:12.581215 1079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:20:12.591151 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 1079
Aug 06 16:20:12 dist-test-slave-t3q3 krb5kdc[32244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033212, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:12.598956 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.583765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:54733 (local address 127.11.245.190:44261)
0806 16:20:12.583985 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:12.583989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:12.585075 (+ 1086us) server_negotiation.cc:407] Connection header received
0806 16:20:12.586746 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:12.586772 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:12.586846 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:12.586944 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:12.588616 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.589231 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.590530 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.590783 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.590891 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.594300 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:12.594305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:12.594307 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:12.594334 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:12.595956 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:12.596623 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:12.596627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:12.596628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:12.596754 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:12.597219 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:12.597224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:12.597228 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:12.597550 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:12.597673 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:12.598568 (+ 895us) server_negotiation.cc:299] Negotiation successful
0806 16:20:12.598696 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 16:20:12.600464 1267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.584235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:54733)
0806 16:20:12.584877 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:12.584930 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:12.586436 (+ 1506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:12.587210 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:12.587219 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:12.587687 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:12.588422 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:12.588441 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.589376 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.589379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:12.590388 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:12.590404 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.591014 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.591021 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:12.591270 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.592304 (+ 1034us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:12.592324 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:12.594085 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:12.596129 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:12.596137 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:12.596161 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:12.596482 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:12.596861 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:12.596864 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:12.596866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:12.597073 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:12.597694 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:12.597704 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:12.598177 (+ 473us) client_negotiation.cc:769] Sending connection context
0806 16:20:12.598472 (+ 295us) client_negotiation.cc:240] Negotiation successful
0806 16:20:12.598966 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":146,"threads_started":1}
I20260806 16:20:12.602015 1265 heartbeater.cc:347] Connected to a master server at 127.11.245.190:44261
I20260806 16:20:12.602087 1268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.592346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52890 (local address 127.11.245.190:44261)
0806 16:20:12.592673 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:12.592678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:12.592904 (+ 226us) server_negotiation.cc:407] Connection header received
0806 16:20:12.593114 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:12.593121 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:12.593177 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:12.593286 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:12.594212 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.594790 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.595753 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.596014 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.596200 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.597762 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:12.597766 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:12.597769 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:12.597796 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:12.599652 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:12.600225 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:12.600229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:12.600231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:12.600329 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:12.601089 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:12.601092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:12.601094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:12.601304 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:12.601576 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:12.601758 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 16:20:12.601895 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":100,"threads_started":1}
I20260806 16:20:12.602349 1265 heartbeater.cc:464] Registering TS with master...
I20260806 16:20:12.603168 1265 heartbeater.cc:511] Master 127.11.245.190:44261 requested a full tablet report, sending...
I20260806 16:20:12.604450 603 ts_manager.cc:194] Registered new tserver with Master: 179757f75c874f76a99c7ce5d1369cdd (127.11.245.131:41759)
I20260806 16:20:12.605096 603 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:54733
I20260806 16:20:12.616508 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:20:12.622017 1268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.617339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52900 (local address 127.11.245.190:44261)
0806 16:20:12.617512 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:12.617516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:12.617746 (+ 230us) server_negotiation.cc:407] Connection header received
0806 16:20:12.617917 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:12.617920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:12.618063 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:12.618173 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:20:12.618485 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.619096 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.620054 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.620391 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.620487 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.620910 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:20:12.621530 (+ 620us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:20:12.621660 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:12.621778 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 16:20:12.621871 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 16:20:12.623068 603 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:52900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:20:12.627496 603 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:20:12.637888 783 tablet_service.cc:1467] Processing CreateTablet for tablet 14d5beb98155451b925f79e609c115c7 (DEFAULT_TABLE table=test-table [id=6e8fbdf596c54442959ba168ae8aee4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:20:12.638864 783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14d5beb98155451b925f79e609c115c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:12.643136 1281 tablet_bootstrap.cc:492] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7: Bootstrap starting.
I20260806 16:20:12.643263 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.636312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:41759 (local address 127.0.0.1:47362)
0806 16:20:12.636670 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:12.636691 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:12.636811 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:12.638296 (+ 1485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:12.638299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:12.638318 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:12.638467 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:12.638475 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.640199 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.640203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:12.641371 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:12.641384 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.642659 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.642663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:12.642791 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.642883 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 16:20:12.642963 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 16:20:12.643077 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":18,"thread_start_us":103,"threads_started":1}
I20260806 16:20:12.643601 1277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.636620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:33079 (local address 127.0.0.1:34152)
0806 16:20:12.636983 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:12.636999 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:12.637105 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:12.638439 (+ 1334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:12.638441 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:12.638461 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:12.638560 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:12.638567 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.639828 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.639835 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:12.641526 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:12.641535 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.642957 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.642960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:12.643079 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.643106 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 16:20:12.643185 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 16:20:12.643421 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":33,"thread_start_us":119,"threads_started":1}
I20260806 16:20:12.643836 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.636643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47362 (local address 127.11.245.131:41759)
0806 16:20:12.637416 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:12.637435 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:12.637466 (+ 31us) server_negotiation.cc:407] Connection header received
0806 16:20:12.637574 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:12.637606 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:12.638097 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:12.638773 (+ 676us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:12.638980 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.640019 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.641524 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.642510 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.642662 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.642866 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:12.643515 (+ 649us) server_negotiation.cc:299] Negotiation successful
0806 16:20:12.643656 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":151,"threads_started":1}
I20260806 16:20:12.644272 1280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.637132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34152 (local address 127.11.245.130:33079)
0806 16:20:12.637572 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:12.637583 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:12.637625 (+ 42us) server_negotiation.cc:407] Connection header received
0806 16:20:12.637722 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:12.637728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:12.637966 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:12.638190 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:12.638712 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.639678 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.641667 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.642644 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.642763 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.642882 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:12.643893 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0806 16:20:12.644055 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":206,"threads_started":1}
I20260806 16:20:12.644727 1281 tablet_bootstrap.cc:654] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:12.646315 1281 tablet_bootstrap.cc:492] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7: No bootstrap required, opened a new log
I20260806 16:20:12.646440 1281 ts_tablet_manager.cc:1397] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:20:12.646989 1281 raft_consensus.cc:348] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:12.647125 1281 raft_consensus.cc:374] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:12.647195 1281 raft_consensus.cc:729] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892b4f4c5a9e45c79763b3ddf35a13e7, State: Initialized, Role: FOLLOWER
I20260806 16:20:12.647397 1281 consensus_queue.cc:260] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:12.647794 1281 ts_tablet_manager.cc:1428] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 16:20:12.648278 975 tablet_service.cc:1467] Processing CreateTablet for tablet 14d5beb98155451b925f79e609c115c7 (DEFAULT_TABLE table=test-table [id=6e8fbdf596c54442959ba168ae8aee4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:20:12.648283 1168 tablet_service.cc:1467] Processing CreateTablet for tablet 14d5beb98155451b925f79e609c115c7 (DEFAULT_TABLE table=test-table [id=6e8fbdf596c54442959ba168ae8aee4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:20:12.649797 975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14d5beb98155451b925f79e609c115c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:12.649821 1168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14d5beb98155451b925f79e609c115c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:12.659169 1287 tablet_bootstrap.cc:492] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd: Bootstrap starting.
I20260806 16:20:12.659348 1288 tablet_bootstrap.cc:492] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343: Bootstrap starting.
I20260806 16:20:12.662324 1287 tablet_bootstrap.cc:654] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:12.662794 1288 tablet_bootstrap.cc:654] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:12.663662 1287 log.cc:824] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:12.663662 1288 log.cc:824] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:12.665966 1287 tablet_bootstrap.cc:492] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd: No bootstrap required, opened a new log
I20260806 16:20:12.665964 1288 tablet_bootstrap.cc:492] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343: No bootstrap required, opened a new log
I20260806 16:20:12.666208 1288 ts_tablet_manager.cc:1397] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260806 16:20:12.666214 1287 ts_tablet_manager.cc:1397] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260806 16:20:12.670053 1288 raft_consensus.cc:348] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:12.670053 1287 raft_consensus.cc:348] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:12.670372 1288 raft_consensus.cc:374] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:12.670367 1287 raft_consensus.cc:374] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:12.670459 1287 raft_consensus.cc:729] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 179757f75c874f76a99c7ce5d1369cdd, State: Initialized, Role: FOLLOWER
I20260806 16:20:12.670480 1288 raft_consensus.cc:729] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac47217cf3b946239b28ecbb1f383343, State: Initialized, Role: FOLLOWER
I20260806 16:20:12.671015 1287 consensus_queue.cc:260] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:12.671015 1288 consensus_queue.cc:260] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:12.671866 1076 heartbeater.cc:503] Master 127.11.245.190:44261 was elected leader, sending a full tablet report...
I20260806 16:20:12.671885 1265 heartbeater.cc:503] Master 127.11.245.190:44261 was elected leader, sending a full tablet report...
I20260806 16:20:12.672339 1288 ts_tablet_manager.cc:1428] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260806 16:20:12.673489 1287 ts_tablet_manager.cc:1428] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
W20260806 16:20:12.826399 872 tablet.cc:2369] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:20:12.830010 1077 tablet.cc:2369] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:20:12.832458 1266 tablet.cc:2369] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:20:12.883904 1296 raft_consensus.cc:482] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:20:12.884341 1296 raft_consensus.cc:504] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:12.886066 1296 leader_election.cc:288] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079), 892b4f4c5a9e45c79763b3ddf35a13e7 (127.11.245.129:33525)
I20260806 16:20:12.891597 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.886814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:43733 (local address 127.11.245.129:33525)
0806 16:20:12.887184 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:12.887204 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:12.887223 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:20:12.887296 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:12.887300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:12.887387 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:12.887550 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:12.887846 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.888663 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.890055 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.890778 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.890897 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.891077 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:12.891347 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 16:20:12.891442 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":126,"threads_started":1}
I20260806 16:20:12.891592 1280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.886584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:48733 (local address 127.11.245.130:33079)
0806 16:20:12.886743 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:12.886748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:12.886766 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:20:12.886983 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:12.886987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:12.887069 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:12.887184 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:12.887518 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.888417 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.890055 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.890695 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.890861 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.890934 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:12.891347 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 16:20:12.891437 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 16:20:12.891593 1267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.886495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:33079 (local address 127.11.245.131:48733)
0806 16:20:12.886656 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:12.886676 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:12.886803 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:12.887213 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:12.887217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:12.887236 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:12.887379 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:12.887384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.888544 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.888547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:12.889874 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:12.889887 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.890906 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.890910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:12.891174 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.891218 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 16:20:12.891311 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 16:20:12.891417 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":68}
I20260806 16:20:12.891603 1297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.886500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:33525 (local address 127.11.245.131:43733)
0806 16:20:12.887030 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:12.887042 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:12.887162 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:12.887572 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:12.887574 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:12.887594 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:12.887697 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:12.887702 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.888776 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.888779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:12.889899 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:12.889906 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.890918 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.890921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:12.891164 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.891216 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 16:20:12.891311 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 16:20:12.891398 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":142,"thread_start_us":165,"threads_started":1}
I20260806 16:20:12.892789 1005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5beb98155451b925f79e609c115c7" candidate_uuid: "179757f75c874f76a99c7ce5d1369cdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac47217cf3b946239b28ecbb1f383343" is_pre_election: true
I20260806 16:20:12.892768 808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5beb98155451b925f79e609c115c7" candidate_uuid: "179757f75c874f76a99c7ce5d1369cdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" is_pre_election: true
I20260806 16:20:12.893270 808 raft_consensus.cc:2454] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 179757f75c874f76a99c7ce5d1369cdd in term 0.
I20260806 16:20:12.893268 1005 raft_consensus.cc:2454] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 179757f75c874f76a99c7ce5d1369cdd in term 0.
I20260806 16:20:12.894029 1103 leader_election.cc:302] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 179757f75c874f76a99c7ce5d1369cdd, 892b4f4c5a9e45c79763b3ddf35a13e7, ac47217cf3b946239b28ecbb1f383343; no voters:
I20260806 16:20:12.894501 1296 raft_consensus.cc:2781] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:20:12.894644 1296 raft_consensus.cc:482] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:20:12.894739 1296 raft_consensus.cc:3037] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:12.896195 1296 raft_consensus.cc:504] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:12.896644 1296 leader_election.cc:288] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [CANDIDATE]: Term 1 election: Requested vote from peers ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079), 892b4f4c5a9e45c79763b3ddf35a13e7 (127.11.245.129:33525)
I20260806 16:20:12.897192 808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5beb98155451b925f79e609c115c7" candidate_uuid: "179757f75c874f76a99c7ce5d1369cdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7"
I20260806 16:20:12.897341 1005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d5beb98155451b925f79e609c115c7" candidate_uuid: "179757f75c874f76a99c7ce5d1369cdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac47217cf3b946239b28ecbb1f383343"
I20260806 16:20:12.897384 808 raft_consensus.cc:3037] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:12.897550 1005 raft_consensus.cc:3037] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:12.899039 1005 raft_consensus.cc:2454] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 179757f75c874f76a99c7ce5d1369cdd in term 1.
I20260806 16:20:12.899039 808 raft_consensus.cc:2454] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 179757f75c874f76a99c7ce5d1369cdd in term 1.
I20260806 16:20:12.899506 1104 leader_election.cc:302] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 179757f75c874f76a99c7ce5d1369cdd, ac47217cf3b946239b28ecbb1f383343; no voters:
I20260806 16:20:12.899863 1296 raft_consensus.cc:2781] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:20:12.900178 1296 raft_consensus.cc:686] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [term 1 LEADER]: Becoming Leader. State: Replica: 179757f75c874f76a99c7ce5d1369cdd, State: Running, Role: LEADER
I20260806 16:20:12.900662 1296 consensus_queue.cc:237] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:12.905141 603 catalog_manager.cc:5495] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd reported cstate change: term changed from 0 to 1, leader changed from <none> to 179757f75c874f76a99c7ce5d1369cdd (127.11.245.131). New cstate: current_term: 1 leader_uuid: "179757f75c874f76a99c7ce5d1369cdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } health_report { overall_health: UNKNOWN } } }
I20260806 16:20:12.945335 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:12.941008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47364 (local address 127.11.245.131:41759)
0806 16:20:12.941184 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:12.941188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:12.941206 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:20:12.941321 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:12.941324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:12.941393 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:12.941500 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:20:12.941782 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.942360 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.943314 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:12.943578 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:12.943677 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:12.944214 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:20:12.944837 (+ 623us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:20:12.944958 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:12.945072 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 16:20:12.945144 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 16:20:12.953444 1005 raft_consensus.cc:1261] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343 [term 1 FOLLOWER]: Refusing update from remote peer 179757f75c874f76a99c7ce5d1369cdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:20:12.953441 808 raft_consensus.cc:1261] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Refusing update from remote peer 179757f75c874f76a99c7ce5d1369cdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:20:12.954293 1296 consensus_queue.cc:1035] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:20:12.954572 1299 consensus_queue.cc:1035] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:20:12.975466 602 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:52900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:20:12.975761 602 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:52900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:20:12.978994 602 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: Sending DeleteTablet for 3 replicas of tablet 14d5beb98155451b925f79e609c115c7
I20260806 16:20:12.979888 783 tablet_service.cc:1514] Processing DeleteTablet for tablet 14d5beb98155451b925f79e609c115c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:20:12 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:39084
I20260806 16:20:12.979929 1168 tablet_service.cc:1514] Processing DeleteTablet for tablet 14d5beb98155451b925f79e609c115c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:20:12 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:47362
I20260806 16:20:12.979969 975 tablet_service.cc:1514] Processing DeleteTablet for tablet 14d5beb98155451b925f79e609c115c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:20:12 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:34152
I20260806 16:20:12.980602 1315 tablet_replica.cc:330] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd: stopping tablet replica
I20260806 16:20:12.980640 1314 tablet_replica.cc:330] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7: stopping tablet replica
I20260806 16:20:12.980806 1314 raft_consensus.cc:2229] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:20:12.981088 1314 raft_consensus.cc:2258] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:20:12.981115 1315 raft_consensus.cc:2229] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:20:12.981211 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 659
I20260806 16:20:12.981791 1315 raft_consensus.cc:2258] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:20:12.984879 1316 tablet_replica.cc:330] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343: stopping tablet replica
I20260806 16:20:12.985334 1316 raft_consensus.cc:2229] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:20:12.985741 1316 raft_consensus.cc:2258] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:20:12.992357 1315 ts_tablet_manager.cc:1905] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:20:12.992357 1316 ts_tablet_manager.cc:1905] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:20:12.995826 1315 ts_tablet_manager.cc:1918] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:20:12.995826 1316 ts_tablet_manager.cc:1918] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:20:12.995965 1316 log.cc:1196] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/14d5beb98155451b925f79e609c115c7
I20260806 16:20:12.995988 1315 log.cc:1196] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/14d5beb98155451b925f79e609c115c7
I20260806 16:20:12.996441 1316 ts_tablet_manager.cc:1939] T 14d5beb98155451b925f79e609c115c7 P ac47217cf3b946239b28ecbb1f383343: Deleting consensus metadata
I20260806 16:20:12.996441 1315 ts_tablet_manager.cc:1939] T 14d5beb98155451b925f79e609c115c7 P 179757f75c874f76a99c7ce5d1369cdd: Deleting consensus metadata
I20260806 16:20:12.998186 581 catalog_manager.cc:4841] TS 179757f75c874f76a99c7ce5d1369cdd (127.11.245.131:41759): tablet 14d5beb98155451b925f79e609c115c7 (table test-table [id=6e8fbdf596c54442959ba168ae8aee4c]) successfully deleted
I20260806 16:20:12.998186 584 catalog_manager.cc:4841] TS ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079): tablet 14d5beb98155451b925f79e609c115c7 (table test-table [id=6e8fbdf596c54442959ba168ae8aee4c]) successfully deleted
W20260806 16:20:13.002110 582 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:33525 (error 108)
I20260806 16:20:13.003080 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 877
I20260806 16:20:13.011885 1277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:13.003383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:33525 (local address 127.0.0.1:39100)
0806 16:20:13.011559 (+ 8176us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:13.011721 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:33525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7975}
W20260806 16:20:13.012432 582 catalog_manager.cc:4584] TS 892b4f4c5a9e45c79763b3ddf35a13e7 (127.11.245.129:33525): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14d5beb98155451b925f79e609c115c7: Network error: Client connection negotiation failed: client connection to 127.11.245.129:33525: connect: Connection refused (error 111)
I20260806 16:20:13.024302 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 1079
I20260806 16:20:13.051584 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 556
I20260806 16:20:13.078831 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:44261
--webserver_interface=127.11.245.190
--webserver_port=40297
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:44261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:20:13.233809 1318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:13.234359 1318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:13.234460 1318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:13.239341 1318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:20:13.239441 1318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:13.239470 1318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:13.239490 1318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:20:13.239571 1318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:20:13.246372 1318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:44261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:44261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=40297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:13.248164 1318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:13.249673 1318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:13.261087 1329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:13.261121 1333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:13.261482 1318 server_base.cc:1034] running on GCE node
W20260806 16:20:13.261089 1326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:13.262332 1318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:13.264099 1318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:13.265343 1318 hybrid_clock.cc:648] HybridClock initialized: now 1786033213265290 us; error 76 us; skew 500 ppm
Aug 06 16:20:13 dist-test-slave-t3q3 krb5kdc[32244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033213, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:13.269280 1318 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:20:13.270646 1318 webserver.cc:460] Webserver started at http://127.11.245.190:40297/ using document root <none> and password file <none>
I20260806 16:20:13.271420 1318 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:13.271708 1318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:13.277130 1318 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:20:13.280431 1343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:13.282258 1318 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:20:13.282507 1318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "16c771abcc6b41eea0cef489181e770c"
format_stamp: "Formatted at 2026-08-06 16:20:10 on dist-test-slave-t3q3"
server_key: "e33e7d4a1f5e2e692861cf7577b6dae0"
server_key_iv: "a1b2a56c423c2eaf703329eaea3923d6"
server_key_version: "encryptionkey@0"
I20260806 16:20:13.283337 1318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:13.304210 1318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:13.308447 1318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:13.308705 1318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:13.318315 1318 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:44261
I20260806 16:20:13.318653 1416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:44261 every 8 connection(s)
I20260806 16:20:13.320391 1318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:20:13.320935 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 1318
I20260806 16:20:13.321523 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:33525
--local_ip_for_outbound_sockets=127.11.245.129
--tserver_master_addrs=127.11.245.190:44261
--webserver_port=33425
--webserver_interface=127.11.245.129
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:20:13.327165 1417 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:20:13.329960 1417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: Bootstrap starting.
I20260806 16:20:13.353322 1417 log.cc:824] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:13.367023 1417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:20:13.367633 1417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: Bootstrap complete.
I20260806 16:20:13.373481 1417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } }
I20260806 16:20:13.374032 1417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16c771abcc6b41eea0cef489181e770c, State: Initialized, Role: FOLLOWER
I20260806 16:20:13.374766 1417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } }
I20260806 16:20:13.375074 1417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:20:13.375209 1417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:20:13.375356 1417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 2 FOLLOWER]: Advancing to term 3
I20260806 16:20:13.378204 1417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } }
I20260806 16:20:13.378706 1417 leader_election.cc:302] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16c771abcc6b41eea0cef489181e770c; no voters:
I20260806 16:20:13.379209 1417 leader_election.cc:288] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 16:20:13.379380 1422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 3 FOLLOWER]: Leader election won for term 3
I20260806 16:20:13.379667 1422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [term 3 LEADER]: Becoming Leader. State: Replica: 16c771abcc6b41eea0cef489181e770c, State: Running, Role: LEADER
I20260806 16:20:13.380085 1422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } }
I20260806 16:20:13.381011 1417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:20:13.382019 1424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16c771abcc6b41eea0cef489181e770c. Latest consensus state: current_term: 3 leader_uuid: "16c771abcc6b41eea0cef489181e770c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } } }
I20260806 16:20:13.382040 1423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "16c771abcc6b41eea0cef489181e770c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c771abcc6b41eea0cef489181e770c" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44261 } } }
I20260806 16:20:13.382180 1424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [sys.catalog]: This master's current role is: LEADER
I20260806 16:20:13.382184 1423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c [sys.catalog]: This master's current role is: LEADER
I20260806 16:20:13.387753 1441 catalog_manager.cc:1259] Loaded cluster ID: 3d526d088355494c8e352afef26824f3
I20260806 16:20:13.387921 1441 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: loading cluster ID for follower catalog manager: success
W20260806 16:20:13.389384 1441 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 16c771abcc6b41eea0cef489181e770c: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260806 16:20:13.389460 1441 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260806 16:20:13.389756 1431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:20:13.393328 1431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:20:13.394047 1431 catalog_manager.cc:1259] Loaded cluster ID: 3d526d088355494c8e352afef26824f3
I20260806 16:20:13.394124 1431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 16:20:13.394842 1431 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7be6c899ad google::LogMessage::Fail() at ??:0
@ 0x7f7be6c8c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f7be6c894de google::LogMessage::Flush() at ??:0
@ 0x7f7be6c8aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7bf15ccfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f7bf15cdd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7bf15ca579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f7bf160e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f7be8edb3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7be8edbc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f7be8edd3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f7be8ecc512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7bea5106db start_thread at ??:0
@ 0x7f7be578761f clone at ??:0
W20260806 16:20:13.518023 1419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:13.518383 1419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:13.518435 1419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:13.522974 1419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:13.523066 1419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:13.523176 1419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:20:13.529567 1419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:33525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=33425
--tserver_master_addrs=127.11.245.190:44261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:13.531281 1419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:13.532768 1419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:13.545753 1481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:13.545846 1479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:13.545789 1478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:13.546059 1419 server_base.cc:1034] running on GCE node
I20260806 16:20:13.546845 1419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:13.547854 1419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:13.549089 1419 hybrid_clock.cc:648] HybridClock initialized: now 1786033213549081 us; error 67 us; skew 500 ppm
Aug 06 16:20:13 dist-test-slave-t3q3 krb5kdc[32244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033213, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:13.552831 1419 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:20:13.554208 1419 webserver.cc:460] Webserver started at http://127.11.245.129:33425/ using document root <none> and password file <none>
I20260806 16:20:13.554968 1419 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:13.555073 1419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:13.560585 1419 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:20:13.563477 1489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:13.565100 1419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:20:13.565243 1419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "892b4f4c5a9e45c79763b3ddf35a13e7"
format_stamp: "Formatted at 2026-08-06 16:20:10 on dist-test-slave-t3q3"
server_key: "fa3ecc6a04c8043816da2417053ca9e9"
server_key_iv: "27a87ac4c1e4249f19dd836ae64ddea0"
server_key_version: "encryptionkey@0"
I20260806 16:20:13.565937 1419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:13.582865 1419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:13.587136 1419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:13.587415 1419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:13.588346 1419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:20:13.589957 1497 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 16:20:13.597337 1419 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 16:20:13.597477 1419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.000s sys 0.001s
I20260806 16:20:13.597621 1419 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 16:20:13.599961 1419 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 16:20:13.600047 1419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 16:20:13.600486 1497 tablet_bootstrap.cc:492] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7: Bootstrap starting.
I20260806 16:20:13.614354 1419 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:33525
I20260806 16:20:13.614384 1650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:33525 every 8 connection(s)
I20260806 16:20:13.615749 1419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:20:13.624941 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 1419
I20260806 16:20:13.625541 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:33079
--local_ip_for_outbound_sockets=127.11.245.130
--tserver_master_addrs=127.11.245.190:44261
--webserver_port=33249
--webserver_interface=127.11.245.130
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:20:13.628511 1497 log.cc:824] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:13.643604 1497 tablet_bootstrap.cc:492] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:20:13.644251 1497 tablet_bootstrap.cc:492] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7: Bootstrap complete.
I20260806 16:20:13.646179 1497 ts_tablet_manager.cc:1397] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7: Time spent bootstrapping tablet: real 0.046s user 0.029s sys 0.015s
I20260806 16:20:13.650522 1497 raft_consensus.cc:348] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:13.651191 1497 raft_consensus.cc:729] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892b4f4c5a9e45c79763b3ddf35a13e7, State: Initialized, Role: FOLLOWER
I20260806 16:20:13.651891 1497 consensus_queue.cc:260] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:13.653348 1497 ts_tablet_manager.cc:1428] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
W20260806 16:20:13.686170 1654 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:20:13.619505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:39027)
0806 16:20:13.620276 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:13.620364 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:13.621965 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:13.684895 (+ 62930us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":173,"threads_started":1}
W20260806 16:20:13.686879 1651 heartbeater.cc:644] Failed to heartbeat to 127.11.245.190:44261 (0 consecutive failures): Network error: Failed to ping master at 127.11.245.190:44261: Client connection negotiation failed: client connection to 127.11.245.190:44261: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 16:20:13.687988 1654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:13.687538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:44431)
0806 16:20:13.687749 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:13.687865 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 16:20:13.688951 1654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:13.688683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:47081)
0806 16:20:13.688819 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:13.688879 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 16:20:13.689210 1651 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:20:13.787042 1656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:13.787465 1656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:13.787523 1656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:13.793009 1656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:13.793144 1656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:13.793221 1656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:20:13.800016 1656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:33079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=33249
--tserver_master_addrs=127.11.245.190:44261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:13.801708 1656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:13.803125 1656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:13.817068 1673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:13.817210 1671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:13.817134 1656 server_base.cc:1034] running on GCE node
W20260806 16:20:13.817310 1667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:13.818193 1656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:13.820154 1656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:13.821405 1656 hybrid_clock.cc:648] HybridClock initialized: now 1786033213821389 us; error 42 us; skew 500 ppm
Aug 06 16:20:13 dist-test-slave-t3q3 krb5kdc[32244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033213, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:13.826493 1656 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:20:13.828147 1656 webserver.cc:460] Webserver started at http://127.11.245.130:33249/ using document root <none> and password file <none>
I20260806 16:20:13.829001 1656 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:13.829088 1656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:13.834844 1656 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:20:13.838181 1680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:13.840266 1656 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:20:13.840445 1656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "ac47217cf3b946239b28ecbb1f383343"
format_stamp: "Formatted at 2026-08-06 16:20:10 on dist-test-slave-t3q3"
server_key: "08928eab00eb915fa60ae1502504d6c0"
server_key_iv: "a6bfd8dfa6781d1935d53ac465ffa13e"
server_key_version: "encryptionkey@0"
I20260806 16:20:13.841257 1656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:13.867182 1656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:13.872838 1656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:13.873154 1656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:13.874018 1656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:20:13.875171 1656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:20:13.875255 1656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:13.875339 1656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:20:13.875396 1656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:13.886830 1656 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:33079
I20260806 16:20:13.886857 1842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:33079 every 8 connection(s)
I20260806 16:20:13.888300 1656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:20:13.890535 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 1656
I20260806 16:20:13.891072 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:41759
--local_ip_for_outbound_sockets=127.11.245.131
--tserver_master_addrs=127.11.245.190:44261
--webserver_port=42203
--webserver_interface=127.11.245.131
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:20:13.894572 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:13.891669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:44067)
0806 16:20:13.892487 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:13.892692 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":165,"threads_started":1}
W20260806 16:20:13.895141 1843 heartbeater.cc:644] Failed to heartbeat to 127.11.245.190:44261 (0 consecutive failures): Network error: Failed to ping master at 127.11.245.190:44261: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
I20260806 16:20:13.896024 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:13.895683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:60067)
0806 16:20:13.895849 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:13.895924 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 16:20:13.896883 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:13.896602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:37725)
0806 16:20:13.896744 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:13.896806 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 16:20:13.897113 1843 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:20:14.044256 1847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:14.044652 1847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:14.044703 1847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:14.049437 1847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:14.049538 1847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:14.049613 1847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:20:14.056123 1847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:41759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=42203
--tserver_master_addrs=127.11.245.190:44261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:14.057716 1847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:14.059126 1847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:14.073184 1852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:14.073413 1847 server_base.cc:1034] running on GCE node
W20260806 16:20:14.073272 1854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:14.073210 1856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:14.074250 1847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:14.075218 1847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:14.076525 1847 hybrid_clock.cc:648] HybridClock initialized: now 1786033214076486 us; error 59 us; skew 500 ppm
Aug 06 16:20:14 dist-test-slave-t3q3 krb5kdc[32244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033214, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:14.080332 1847 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:20:14.081722 1847 webserver.cc:460] Webserver started at http://127.11.245.131:42203/ using document root <none> and password file <none>
I20260806 16:20:14.082638 1847 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:14.082751 1847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:14.089068 1847 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:20:14.092280 1866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:14.094064 1847 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:20:14.094257 1847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "179757f75c874f76a99c7ce5d1369cdd"
format_stamp: "Formatted at 2026-08-06 16:20:11 on dist-test-slave-t3q3"
server_key: "091a32b073328a3d913582e2675c92c5"
server_key_iv: "c1ab866f22761cf526ac2f8354caff16"
server_key_version: "encryptionkey@0"
I20260806 16:20:14.095059 1847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:14.122697 1847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:14.128958 1847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:14.129269 1847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:14.130194 1847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:20:14.132006 1847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:20:14.132133 1847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 16:20:14.132231 1847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:20:14.132284 1847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:14.146747 1847 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:41759
I20260806 16:20:14.146768 2024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:41759 every 8 connection(s)
I20260806 16:20:14.148373 1847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:20:14.154379 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 1847
I20260806 16:20:14.154557 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:14.151481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:37331)
0806 16:20:14.152581 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:14.152756 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":756,"thread_start_us":156,"threads_started":1}
W20260806 16:20:14.155365 2026 heartbeater.cc:644] Failed to heartbeat to 127.11.245.190:44261 (0 consecutive failures): Network error: Failed to ping master at 127.11.245.190:44261: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
I20260806 16:20:14.156201 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:14.155888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:46421)
0806 16:20:14.156026 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:14.156115 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 16:20:14.157191 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:14.156906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:49727)
0806 16:20:14.157065 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:14.157125 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 16:20:14.157459 2026 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 16:20:14.690639 2035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:14.690011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:47615)
0806 16:20:14.690408 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:14.690490 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":114,"threads_started":1}
I20260806 16:20:14.898797 2036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:14.898146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:40485)
0806 16:20:14.898545 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:14.898627 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":105,"threads_started":1}
I20260806 16:20:15.074975 2037 raft_consensus.cc:482] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:20:15.075322 2037 raft_consensus.cc:504] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:15.077114 2037 leader_election.cc:288] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 179757f75c874f76a99c7ce5d1369cdd (127.11.245.131:41759), ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079)
Aug 06 16:20:15 dist-test-slave-t3q3 krb5kdc[32244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033213, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.130@KRBTEST.COM
Aug 06 16:20:15 dist-test-slave-t3q3 krb5kdc[32244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033213, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.131@KRBTEST.COM
I20260806 16:20:15.093763 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:15.077434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:41759 (local address 127.11.245.129:53789)
0806 16:20:15.078046 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:15.078062 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:15.078180 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:15.080188 (+ 2008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:15.080191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:15.080689 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:15.081253 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:15.081271 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:15.082794 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:15.082797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:15.083868 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:15.083879 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:15.084716 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:15.084720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:15.084970 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:15.085793 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:15.085813 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:15.087951 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:15.090651 (+ 2700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:15.090659 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:15.090677 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:15.091056 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:15.091441 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:15.091444 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:15.091446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:15.091635 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:15.092539 (+ 904us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:15.092542 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:15.092836 (+ 294us) client_negotiation.cc:769] Sending connection context
0806 16:20:15.093078 (+ 242us) client_negotiation.cc:240] Negotiation successful
0806 16:20:15.093541 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":239,"spinlock_wait_cycles":13056,"thread_start_us":147,"threads_started":1}
I20260806 16:20:15.093780 2039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:15.077617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:53789 (local address 127.11.245.131:41759)
0806 16:20:15.078166 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:15.078183 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:15.078235 (+ 52us) server_negotiation.cc:407] Connection header received
0806 16:20:15.079066 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:15.079074 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:15.079547 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:15.079865 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:15.081526 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:15.082643 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:15.084017 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:15.084567 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:15.084766 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:15.088221 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:15.088225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:15.088243 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:15.088284 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:15.090498 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:15.091190 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:15.091201 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:15.091206 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:15.091336 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:15.091739 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:15.091743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:15.091745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:15.092401 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:15.092568 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:15.093201 (+ 633us) server_negotiation.cc:299] Negotiation successful
0806 16:20:15.093531 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":73,"threads_started":1}
I20260806 16:20:15.093780 2040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:15.077635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:37489 (local address 127.11.245.130:33079)
0806 16:20:15.078166 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:15.078194 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:15.078243 (+ 49us) server_negotiation.cc:407] Connection header received
0806 16:20:15.079106 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:15.079113 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:15.079546 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:15.079917 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:15.081507 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:15.082591 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:15.083973 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:15.084567 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:15.084767 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:15.088192 (+ 3425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:15.088200 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:15.088221 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:15.088267 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:15.090577 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:15.091189 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:15.091203 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:15.091208 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:15.091337 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:15.091753 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:15.091756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:15.091758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:15.092396 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:15.092567 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:15.093203 (+ 636us) server_negotiation.cc:299] Negotiation successful
0806 16:20:15.093557 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":173,"threads_started":1}
I20260806 16:20:15.093808 2035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:15.077385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:33079 (local address 127.11.245.129:37489)
0806 16:20:15.077593 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:15.077610 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:15.077748 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:15.080007 (+ 2259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:15.080014 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:15.080677 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:15.081253 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:15.081271 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:15.082750 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:15.082759 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:15.083837 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:15.083849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:15.084716 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:15.084720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:15.084968 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:15.085842 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:15.085855 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:15.087845 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:15.090705 (+ 2860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:15.090710 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:15.090712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:15.091062 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:15.091452 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:15.091455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:15.091457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:15.091645 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:15.092530 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:15.092537 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:15.092836 (+ 299us) client_negotiation.cc:769] Sending connection context
0806 16:20:15.093078 (+ 242us) client_negotiation.cc:240] Negotiation successful
0806 16:20:15.093636 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":359,"spinlock_wait_cycles":53632}
W20260806 16:20:15.097757 1490 leader_election.cc:341] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 179757f75c874f76a99c7ce5d1369cdd (127.11.245.131:41759): Not found: Tablet not found: 14d5beb98155451b925f79e609c115c7
W20260806 16:20:15.098178 1492 leader_election.cc:341] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079): Not found: Tablet not found: 14d5beb98155451b925f79e609c115c7
I20260806 16:20:15.098278 1492 leader_election.cc:302] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 892b4f4c5a9e45c79763b3ddf35a13e7; no voters: 179757f75c874f76a99c7ce5d1369cdd, ac47217cf3b946239b28ecbb1f383343
I20260806 16:20:15.098652 2037 raft_consensus.cc:2726] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:20:15.158773 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:15.158173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:35129)
0806 16:20:15.158555 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:15.158631 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":95,"threads_started":1}
I20260806 16:20:15.692584 2042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:15.691906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:41527)
0806 16:20:15.692365 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:15.692445 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":166,"threads_started":1}
I20260806 16:20:15.900728 2043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:15.900108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:53835)
0806 16:20:15.900456 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:15.900586 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":111,"threads_started":1}
I20260806 16:20:16.161273 2045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:16.160547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:37255)
0806 16:20:16.161003 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:16.161092 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":153,"threads_started":1}
I20260806 16:20:16.694412 2047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:16.693757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:52541)
0806 16:20:16.694204 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:16.694278 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":143,"threads_started":1}
I20260806 16:20:16.882642 2050 raft_consensus.cc:482] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:20:16.882858 2050 raft_consensus.cc:504] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:16.883569 2050 leader_election.cc:288] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 179757f75c874f76a99c7ce5d1369cdd (127.11.245.131:41759), ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079)
W20260806 16:20:16.885119 1490 leader_election.cc:341] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 179757f75c874f76a99c7ce5d1369cdd (127.11.245.131:41759): Not found: Tablet not found: 14d5beb98155451b925f79e609c115c7
W20260806 16:20:16.885254 1492 leader_election.cc:341] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079): Not found: Tablet not found: 14d5beb98155451b925f79e609c115c7
I20260806 16:20:16.885424 1492 leader_election.cc:302] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 892b4f4c5a9e45c79763b3ddf35a13e7; no voters: 179757f75c874f76a99c7ce5d1369cdd, ac47217cf3b946239b28ecbb1f383343
I20260806 16:20:16.885622 2050 raft_consensus.cc:2726] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:20:16.902503 2051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:16.901814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:41091)
0806 16:20:16.902271 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:16.902353 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":134,"threads_started":1}
I20260806 16:20:17.163163 2052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:17.162573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:55183)
0806 16:20:17.162935 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:17.163017 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":105,"threads_started":1}
I20260806 16:20:17.696405 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:17.695453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:33071)
0806 16:20:17.695821 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:17.695918 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":111,"threads_started":1}
I20260806 16:20:17.904594 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:17.903905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:59439)
0806 16:20:17.904361 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:17.904435 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":207,"threads_started":1}
I20260806 16:20:18.165073 2057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:18.164433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:46417)
0806 16:20:18.164819 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:18.164913 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":111,"threads_started":1}
I20260806 16:20:18.698552 2058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:18.697828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:59555)
0806 16:20:18.698266 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:18.698396 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":107,"threads_started":1}
I20260806 16:20:18.906479 2061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:18.905800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:43521)
0806 16:20:18.906215 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:18.906309 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":110,"threads_started":1}
I20260806 16:20:19.167042 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:19.166416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:34525)
0806 16:20:19.166798 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:19.166896 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":131,"threads_started":1}
I20260806 16:20:19.546180 2063 raft_consensus.cc:482] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:20:19.546444 2063 raft_consensus.cc:504] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:19.547175 2063 leader_election.cc:288] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 179757f75c874f76a99c7ce5d1369cdd (127.11.245.131:41759), ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079)
W20260806 16:20:19.548297 1492 leader_election.cc:341] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079): Not found: Tablet not found: 14d5beb98155451b925f79e609c115c7
W20260806 16:20:19.548422 1490 leader_election.cc:341] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 179757f75c874f76a99c7ce5d1369cdd (127.11.245.131:41759): Not found: Tablet not found: 14d5beb98155451b925f79e609c115c7
I20260806 16:20:19.548552 1490 leader_election.cc:302] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 892b4f4c5a9e45c79763b3ddf35a13e7; no voters: 179757f75c874f76a99c7ce5d1369cdd, ac47217cf3b946239b28ecbb1f383343
I20260806 16:20:19.548743 2063 raft_consensus.cc:2726] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:20:19.700349 2064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:19.699639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:38371)
0806 16:20:19.700018 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:19.700150 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":113,"threads_started":1}
I20260806 16:20:19.908351 2065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:19.907667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:57045)
0806 16:20:19.908044 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:19.908184 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":111,"threads_started":1}
I20260806 16:20:20.168849 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:20.168251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:54499)
0806 16:20:20.168623 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:20.168705 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":104,"threads_started":1}
I20260806 16:20:20.701964 2067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:20.701401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:45995)
0806 16:20:20.701750 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:20.701819 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":132,"threads_started":1}
I20260806 16:20:20.910233 2068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:20.909522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:32969)
0806 16:20:20.910009 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:20.910092 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":140,"threads_started":1}
I20260806 16:20:21.170636 2070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:21.169990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:54337)
0806 16:20:21.170410 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:21.170507 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":103,"threads_started":1}
I20260806 16:20:21.703822 2071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:21.703164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:48911)
0806 16:20:21.703600 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:21.703682 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":141,"threads_started":1}
I20260806 16:20:21.912269 2072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:21.911556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:47175)
0806 16:20:21.911970 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:21.912055 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":120,"threads_started":1}
I20260806 16:20:22.172461 2073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:22.171758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:50199)
0806 16:20:22.172231 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:22.172315 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":114,"threads_started":1}
I20260806 16:20:22.213418 2075 raft_consensus.cc:482] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:20:22.213620 2075 raft_consensus.cc:504] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:22.214294 2075 leader_election.cc:288] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 179757f75c874f76a99c7ce5d1369cdd (127.11.245.131:41759), ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079)
W20260806 16:20:22.215621 1490 leader_election.cc:341] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 179757f75c874f76a99c7ce5d1369cdd (127.11.245.131:41759): Not found: Tablet not found: 14d5beb98155451b925f79e609c115c7
W20260806 16:20:22.215994 1492 leader_election.cc:341] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079): Not found: Tablet not found: 14d5beb98155451b925f79e609c115c7
I20260806 16:20:22.216197 1492 leader_election.cc:302] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 892b4f4c5a9e45c79763b3ddf35a13e7; no voters: 179757f75c874f76a99c7ce5d1369cdd, ac47217cf3b946239b28ecbb1f383343
I20260806 16:20:22.216617 2075 raft_consensus.cc:2726] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:20:22.705781 2076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:22.705170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:49001)
0806 16:20:22.705547 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:22.705633 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":153,"threads_started":1}
I20260806 16:20:22.914474 2078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:22.913745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:48485)
0806 16:20:22.914208 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:22.914288 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":148,"threads_started":1}
I20260806 16:20:23.174371 2079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:23.173722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:54749)
0806 16:20:23.174119 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:23.174195 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":145,"threads_started":1}
I20260806 16:20:23.707791 2080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:23.707189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:59249)
0806 16:20:23.707581 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:23.707659 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":141,"threads_started":1}
I20260806 16:20:23.916896 2081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:23.916278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:48489)
0806 16:20:23.916659 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:23.916736 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":107,"threads_started":1}
I20260806 16:20:24.176502 2082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:24.175890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:48285)
0806 16:20:24.176298 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:24.176363 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":165,"threads_started":1}
I20260806 16:20:24.709357 2084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:24.708815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:57817)
0806 16:20:24.709160 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:24.709228 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":101,"threads_started":1}
I20260806 16:20:24.918926 2086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:24.918263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:33051)
0806 16:20:24.918690 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:24.918777 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":122,"threads_started":1}
I20260806 16:20:25.178299 2087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:25.177653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:53577)
0806 16:20:25.178075 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:25.178161 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":125,"threads_started":1}
I20260806 16:20:25.711218 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:25.710549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:56051)
0806 16:20:25.710987 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:25.711078 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":124,"threads_started":1}
I20260806 16:20:25.920915 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:25.920280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:45201)
0806 16:20:25.920697 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:25.920774 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":143,"threads_started":1}
I20260806 16:20:26.180030 2090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:26.179453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:59259)
0806 16:20:26.179827 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:26.179903 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":158,"threads_started":1}
I20260806 16:20:26.713487 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:26.712791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:42041)
0806 16:20:26.713236 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:26.713349 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":111,"threads_started":1}
I20260806 16:20:26.922814 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:26.922196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:58197)
0806 16:20:26.922596 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:26.922679 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":108,"threads_started":1}
I20260806 16:20:27.182199 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:27.181478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:55517)
0806 16:20:27.181954 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:27.182042 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":186,"threads_started":1}
I20260806 16:20:27.715966 2095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:27.714810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:35387)
0806 16:20:27.715591 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:27.715715 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":165,"threads_started":1}
I20260806 16:20:27.885027 2097 raft_consensus.cc:482] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:20:27.885270 2097 raft_consensus.cc:504] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179757f75c874f76a99c7ce5d1369cdd" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41759 } } peers { permanent_uuid: "ac47217cf3b946239b28ecbb1f383343" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 33079 } } peers { permanent_uuid: "892b4f4c5a9e45c79763b3ddf35a13e7" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33525 } }
I20260806 16:20:27.885941 2097 leader_election.cc:288] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 179757f75c874f76a99c7ce5d1369cdd (127.11.245.131:41759), ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079)
W20260806 16:20:27.886969 1492 leader_election.cc:341] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ac47217cf3b946239b28ecbb1f383343 (127.11.245.130:33079): Not found: Tablet not found: 14d5beb98155451b925f79e609c115c7
W20260806 16:20:27.887111 1490 leader_election.cc:341] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 179757f75c874f76a99c7ce5d1369cdd (127.11.245.131:41759): Not found: Tablet not found: 14d5beb98155451b925f79e609c115c7
I20260806 16:20:27.887212 1490 leader_election.cc:302] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 892b4f4c5a9e45c79763b3ddf35a13e7; no voters: 179757f75c874f76a99c7ce5d1369cdd, ac47217cf3b946239b28ecbb1f383343
I20260806 16:20:27.887394 2097 raft_consensus.cc:2726] T 14d5beb98155451b925f79e609c115c7 P 892b4f4c5a9e45c79763b3ddf35a13e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:20:27.924604 2098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:27.924005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:49693)
0806 16:20:27.924400 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:27.924473 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":183,"threads_started":1}
I20260806 16:20:28.184122 2099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:28.183413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:59329)
0806 16:20:28.183821 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:28.183903 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":139,"threads_started":1}
I20260806 16:20:28.718091 2102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:28.717543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.129:55309)
0806 16:20:28.717879 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:28.717960 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":103,"threads_started":1}
I20260806 16:20:28.926532 2104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:28.925947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.130:48205)
0806 16:20:28.926300 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:28.926399 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":104,"threads_started":1}
I20260806 16:20:29.161217 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 1419
I20260806 16:20:29.187465 2106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:29.186918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:44261 (local address 127.11.245.131:52687)
0806 16:20:29.187255 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:29.187335 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:44261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":117,"threads_started":1}
I20260806 16:20:29.197299 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 1656
I20260806 16:20:29.228718 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 1847
2026-08-06T16:20:29Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (20741 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:20:29 dist-test-slave-t3q3 krb5kdc[2109](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:20:29 dist-test-slave-t3q3 krb5kdc[2109](info): set up 2 sockets
Aug 06 16:20:29 dist-test-slave-t3q3 krb5kdc[2109](info): commencing operation
krb5kdc: starting...
W20260806 16:20:30.561829 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.004s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:20:30 dist-test-slave-t3q3 krb5kdc[2109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:20:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:20:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:20:30.752048 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:46857
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:46857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:20:30.911046 2127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:30.911646 2127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:30.911731 2127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:30.916841 2127 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 16:20:30.916937 2127 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 16:20:30.916955 2127 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 16:20:30.916970 2127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:20:30.916991 2127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:30.917011 2127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:30.917028 2127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:20:30.917047 2127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:20:30.923728 2127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:46857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:46857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:30.925357 2127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:30.926855 2127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:30.939097 2136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:30.939175 2127 server_base.cc:1034] running on GCE node
W20260806 16:20:30.939181 2145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:30.939357 2140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:30.940822 2127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:30.942562 2127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:30.943955 2127 hybrid_clock.cc:648] HybridClock initialized: now 1786033230943890 us; error 88 us; skew 500 ppm
Aug 06 16:20:30 dist-test-slave-t3q3 krb5kdc[2109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033230, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:30.948011 2127 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:20:30.949450 2127 webserver.cc:460] Webserver started at http://127.11.245.190:36501/ using document root <none> and password file <none>
I20260806 16:20:30.950253 2127 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:30.950330 2127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:30.950598 2127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:20:30.954015 2127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "36797ff4d85440b78deada47e23ef7ce"
format_stamp: "Formatted at 2026-08-06 16:20:30 on dist-test-slave-t3q3"
server_key: "d8c2ff7a3f2f698df5ecf8b6ef9399de"
server_key_iv: "cbb0558cd6f955ae4eaf004917ead08a"
server_key_version: "encryptionkey@0"
I20260806 16:20:30.954879 2127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "36797ff4d85440b78deada47e23ef7ce"
format_stamp: "Formatted at 2026-08-06 16:20:30 on dist-test-slave-t3q3"
server_key: "d8c2ff7a3f2f698df5ecf8b6ef9399de"
server_key_iv: "cbb0558cd6f955ae4eaf004917ead08a"
server_key_version: "encryptionkey@0"
I20260806 16:20:30.960608 2127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 16:20:30.964493 2153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:30.966665 2127 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:20:30.966857 2127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "36797ff4d85440b78deada47e23ef7ce"
format_stamp: "Formatted at 2026-08-06 16:20:30 on dist-test-slave-t3q3"
server_key: "d8c2ff7a3f2f698df5ecf8b6ef9399de"
server_key_iv: "cbb0558cd6f955ae4eaf004917ead08a"
server_key_version: "encryptionkey@0"
I20260806 16:20:30.967020 2127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:30.987792 2127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:30.992962 2127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:30.993213 2127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:31.004567 2127 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:46857
I20260806 16:20:31.004992 2221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:46857 every 8 connection(s)
I20260806 16:20:31.006274 2127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:20:31.007529 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 2127
I20260806 16:20:31.007695 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:20:31.008208 12246 external_mini_cluster.cc:1428] Setting key f2e8d550150543a7dfc6d29cc5b9b3f4
I20260806 16:20:31.011454 2222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:31.021986 2222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Bootstrap starting.
Aug 06 16:20:31 dist-test-slave-t3q3 krb5kdc[2109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:31.026204 2222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:31.027513 2222 log.cc:824] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:31.030768 2222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: No bootstrap required, opened a new log
I20260806 16:20:31.031538 2226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:31.010370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33618 (local address 127.11.245.190:46857)
0806 16:20:31.011037 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:31.011063 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:31.011123 (+ 60us) server_negotiation.cc:407] Connection header received
0806 16:20:31.012291 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:31.012326 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:31.012922 (+ 596us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:31.013883 (+ 961us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:31.014832 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:31.016587 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:31.017874 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:31.018664 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:31.018878 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:31.023942 (+ 5064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:31.023951 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:31.023973 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:31.024030 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:31.026558 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:31.027321 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:31.027330 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:31.027338 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:31.027518 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:31.027964 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:31.027968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:31.027970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:31.028770 (+ 800us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:31.029011 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:31.029902 (+ 891us) server_negotiation.cc:299] Negotiation successful
0806 16:20:31.030355 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":197,"threads_started":1}
I20260806 16:20:31.036303 2222 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } }
I20260806 16:20:31.036705 2222 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:31.036772 2222 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36797ff4d85440b78deada47e23ef7ce, State: Initialized, Role: FOLLOWER
I20260806 16:20:31.037377 2222 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } }
I20260806 16:20:31.037613 2222 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:20:31.037724 2222 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:20:31.037866 2222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:31.039186 2222 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } }
I20260806 16:20:31.039657 2222 leader_election.cc:302] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36797ff4d85440b78deada47e23ef7ce; no voters:
I20260806 16:20:31.040170 2222 leader_election.cc:288] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:20:31.040311 2229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:20:31.040838 2229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 1 LEADER]: Becoming Leader. State: Replica: 36797ff4d85440b78deada47e23ef7ce, State: Running, Role: LEADER
I20260806 16:20:31.041366 2229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } }
I20260806 16:20:31.042003 2222 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:20:31.044011 2230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36797ff4d85440b78deada47e23ef7ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } } }
I20260806 16:20:31.043977 2232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36797ff4d85440b78deada47e23ef7ce. Latest consensus state: current_term: 1 leader_uuid: "36797ff4d85440b78deada47e23ef7ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } } }
I20260806 16:20:31.044214 2232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [sys.catalog]: This master's current role is: LEADER
I20260806 16:20:31.044214 2230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [sys.catalog]: This master's current role is: LEADER
I20260806 16:20:31.044991 2245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:20:31.049746 2245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:20:31.059343 2245 catalog_manager.cc:1347] Generated new cluster ID: 3bacf4970a3c45c88a923244fd5637e3
I20260806 16:20:31.059484 2245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:20:31.086360 2245 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:20:31.087709 2245 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:20:31.104626 2245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Generated new TSK 0
I20260806 16:20:31.105796 2245 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:20:31.156631 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:46857
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:20:31.330809 2260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:31.331216 2260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:31.331272 2260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:31.337069 2260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:31.337184 2260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:31.337261 2260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:20:31.344545 2260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:31.346323 2260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:31.348094 2260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:31.361881 2269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:31.361881 2268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:31.361881 2271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:31.362221 2260 server_base.cc:1034] running on GCE node
I20260806 16:20:31.362862 2260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:31.363765 2260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:31.365092 2260 hybrid_clock.cc:648] HybridClock initialized: now 1786033231365043 us; error 66 us; skew 500 ppm
Aug 06 16:20:31 dist-test-slave-t3q3 krb5kdc[2109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033231, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:31.368960 2260 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:20:31.370424 2260 webserver.cc:460] Webserver started at http://127.11.245.129:41653/ using document root <none> and password file <none>
I20260806 16:20:31.371317 2260 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:31.371408 2260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:31.371879 2260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:20:31.375168 2260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "732296a961b14967a5e2a6505fba1ba5"
format_stamp: "Formatted at 2026-08-06 16:20:31 on dist-test-slave-t3q3"
server_key: "d3f588f0a1b8d17aba4282ba855a87ab"
server_key_iv: "cbd0af4711cd3e570674beabbf879ce9"
server_key_version: "encryptionkey@0"
I20260806 16:20:31.376015 2260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "732296a961b14967a5e2a6505fba1ba5"
format_stamp: "Formatted at 2026-08-06 16:20:31 on dist-test-slave-t3q3"
server_key: "d3f588f0a1b8d17aba4282ba855a87ab"
server_key_iv: "cbd0af4711cd3e570674beabbf879ce9"
server_key_version: "encryptionkey@0"
I20260806 16:20:31.382259 2260 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 16:20:31.386328 2283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:31.388484 2260 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:20:31.388725 2260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "732296a961b14967a5e2a6505fba1ba5"
format_stamp: "Formatted at 2026-08-06 16:20:31 on dist-test-slave-t3q3"
server_key: "d3f588f0a1b8d17aba4282ba855a87ab"
server_key_iv: "cbd0af4711cd3e570674beabbf879ce9"
server_key_version: "encryptionkey@0"
I20260806 16:20:31.388926 2260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:31.403272 2260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:31.407683 2260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:31.408007 2260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:31.408921 2260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:20:31.410102 2260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:20:31.410209 2260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:31.410320 2260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:20:31.410372 2260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 16:20:31.425172 2260 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:34733
I20260806 16:20:31.425195 2448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:34733 every 8 connection(s)
I20260806 16:20:31.426604 2260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:20:31.431231 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 2260
I20260806 16:20:31.431574 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:20:31.432026 12246 external_mini_cluster.cc:1428] Setting key f9dfa2da8b92fb509068a890af70ad81
Aug 06 16:20:31 dist-test-slave-t3q3 krb5kdc[2109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033231, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:20:31.449765 2226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:31.429371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:52215 (local address 127.11.245.190:46857)
0806 16:20:31.429600 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:31.429612 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:31.430816 (+ 1204us) server_negotiation.cc:407] Connection header received
0806 16:20:31.434197 (+ 3381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:31.434202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:31.434601 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:31.434709 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:31.436617 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:31.437283 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:31.439758 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:31.440048 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:31.440221 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:31.444281 (+ 4060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:31.444287 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:31.444313 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:31.444356 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:31.446564 (+ 2208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:31.447410 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:31.447418 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:31.447423 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:31.447533 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:31.448041 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:31.448045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:31.448046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:31.448475 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:31.448585 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:31.449382 (+ 797us) server_negotiation.cc:299] Negotiation successful
0806 16:20:31.449593 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
I20260806 16:20:31.451191 2451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:31.429893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:52215)
0806 16:20:31.430601 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:31.430661 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:31.433857 (+ 3196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:31.435005 (+ 1148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:31.435016 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:31.435573 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:31.436373 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:31.436395 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:31.437461 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:31.437476 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:31.439561 (+ 2085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:31.439582 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:31.440326 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:31.440334 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:31.440633 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:31.441549 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:31.441585 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:31.443951 (+ 2366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:31.446757 (+ 2806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:31.446765 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:31.446790 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:31.447231 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:31.447667 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:31.447670 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:31.447672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:31.447869 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:31.448627 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:31.448635 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:31.449032 (+ 397us) client_negotiation.cc:769] Sending connection context
0806 16:20:31.449254 (+ 222us) client_negotiation.cc:240] Negotiation successful
0806 16:20:31.449771 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":173,"threads_started":1}
I20260806 16:20:31.453187 2449 heartbeater.cc:347] Connected to a master server at 127.11.245.190:46857
I20260806 16:20:31.453588 2449 heartbeater.cc:464] Registering TS with master...
I20260806 16:20:31.454335 2449 heartbeater.cc:511] Master 127.11.245.190:46857 requested a full tablet report, sending...
I20260806 16:20:31.456724 2177 ts_manager.cc:194] Registered new tserver with Master: 732296a961b14967a5e2a6505fba1ba5 (127.11.245.129:34733)
I20260806 16:20:31.458760 2177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:52215
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:20:31.472151 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:46857
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:20:31.636468 2456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:31.636861 2456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:31.636920 2456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:31.641497 2456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:31.641623 2456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:31.641757 2456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:20:31.648285 2456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:31.650184 2456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:31.652422 2456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:31.666879 2467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:31.666867 2466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:31.666877 2470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:31.666963 2456 server_base.cc:1034] running on GCE node
I20260806 16:20:31.667713 2456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:31.668744 2456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:31.670003 2456 hybrid_clock.cc:648] HybridClock initialized: now 1786033231669956 us; error 61 us; skew 500 ppm
Aug 06 16:20:31 dist-test-slave-t3q3 krb5kdc[2109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033231, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:31.674423 2456 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:20:31.675961 2456 webserver.cc:460] Webserver started at http://127.11.245.130:45981/ using document root <none> and password file <none>
I20260806 16:20:31.676903 2456 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:31.677024 2456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:31.677357 2456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:20:31.680701 2456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "51943d5d2bb04e778a7b24de28590394"
format_stamp: "Formatted at 2026-08-06 16:20:31 on dist-test-slave-t3q3"
server_key: "82da88dc13669f9347fb54acfec7b349"
server_key_iv: "1a32a126b8660a51ed8817a1a174c4dc"
server_key_version: "encryptionkey@0"
I20260806 16:20:31.681540 2456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "51943d5d2bb04e778a7b24de28590394"
format_stamp: "Formatted at 2026-08-06 16:20:31 on dist-test-slave-t3q3"
server_key: "82da88dc13669f9347fb54acfec7b349"
server_key_iv: "1a32a126b8660a51ed8817a1a174c4dc"
server_key_version: "encryptionkey@0"
I20260806 16:20:31.687372 2456 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:20:31.691802 2479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:31.695135 2456 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260806 16:20:31.695379 2456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "51943d5d2bb04e778a7b24de28590394"
format_stamp: "Formatted at 2026-08-06 16:20:31 on dist-test-slave-t3q3"
server_key: "82da88dc13669f9347fb54acfec7b349"
server_key_iv: "1a32a126b8660a51ed8817a1a174c4dc"
server_key_version: "encryptionkey@0"
I20260806 16:20:31.695595 2456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:31.721130 2456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:31.727061 2456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:31.727401 2456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:31.728595 2456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:20:31.729990 2456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:20:31.730083 2456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:31.730181 2456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:20:31.730232 2456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:31.744315 2456 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:44749
I20260806 16:20:31.744360 2639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:44749 every 8 connection(s)
I20260806 16:20:31.745850 2456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:20:31.748355 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 2456
I20260806 16:20:31.748519 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:20:31.748878 12246 external_mini_cluster.cc:1428] Setting key a8f0a2f6394cb5b96dd17e86d4ed9963
Aug 06 16:20:31 dist-test-slave-t3q3 krb5kdc[2109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033231, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
I20260806 16:20:31.767928 2226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:31.751169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:36577 (local address 127.11.245.190:46857)
0806 16:20:31.751446 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:31.751450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:31.751476 (+ 26us) server_negotiation.cc:407] Connection header received
0806 16:20:31.753123 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:31.753126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:31.753192 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:31.753289 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:31.755375 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:31.756478 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:31.758688 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:31.758984 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:31.759101 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:31.763199 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:31.763205 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:31.763210 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:31.763255 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:31.765185 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:31.765850 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:31.765854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:31.765856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:31.765944 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:31.766436 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:31.766440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:31.766441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:31.766699 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:31.766791 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:31.767595 (+ 804us) server_negotiation.cc:299] Negotiation successful
0806 16:20:31.767759 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
I20260806 16:20:31.769328 2645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:31.749401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:36577)
0806 16:20:31.750115 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:31.750175 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:31.752732 (+ 2557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:31.753621 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:31.753637 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:31.754390 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:31.755149 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:31.755170 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:31.756685 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:31.756694 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:31.758480 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:31.758504 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:31.759238 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:31.759246 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:31.759566 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:31.760534 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:31.760566 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:31.762884 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:31.765347 (+ 2463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:31.765355 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:31.765378 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:31.765731 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:31.766086 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:31.766092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:31.766097 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:31.766309 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:31.766827 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:31.766834 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:31.767203 (+ 369us) client_negotiation.cc:769] Sending connection context
0806 16:20:31.767489 (+ 286us) client_negotiation.cc:240] Negotiation successful
0806 16:20:31.767893 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":182,"threads_started":1}
I20260806 16:20:31.771314 2641 heartbeater.cc:347] Connected to a master server at 127.11.245.190:46857
I20260806 16:20:31.771728 2641 heartbeater.cc:464] Registering TS with master...
I20260806 16:20:31.772687 2641 heartbeater.cc:511] Master 127.11.245.190:46857 requested a full tablet report, sending...
I20260806 16:20:31.774487 2177 ts_manager.cc:194] Registered new tserver with Master: 51943d5d2bb04e778a7b24de28590394 (127.11.245.130:44749)
I20260806 16:20:31.775570 2177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:36577
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:20:31.793146 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:46857
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:20:31.964413 2648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:31.964843 2648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:31.964901 2648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:31.969813 2648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:31.969915 2648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:31.970022 2648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:20:31.977445 2648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:31.979025 2648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:31.980409 2648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:31.993063 2656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:31.993196 2658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:31.993065 2655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:31.994014 2648 server_base.cc:1034] running on GCE node
I20260806 16:20:31.994619 2648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:31.995584 2648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:31.996797 2648 hybrid_clock.cc:648] HybridClock initialized: now 1786033231996751 us; error 75 us; skew 500 ppm
Aug 06 16:20:31 dist-test-slave-t3q3 krb5kdc[2109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033231, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:32.000725 2648 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:20:32.002261 2648 webserver.cc:460] Webserver started at http://127.11.245.131:33937/ using document root <none> and password file <none>
I20260806 16:20:32.003159 2648 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:32.003280 2648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:32.003607 2648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:20:32.007233 2648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "175ae1a9d17341c09827d6d11d712e75"
format_stamp: "Formatted at 2026-08-06 16:20:32 on dist-test-slave-t3q3"
server_key: "2048e5e336187b2be8e82d3efd0ff589"
server_key_iv: "ee445ec19ef5213a2c151fc88676087a"
server_key_version: "encryptionkey@0"
I20260806 16:20:32.008170 2648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "175ae1a9d17341c09827d6d11d712e75"
format_stamp: "Formatted at 2026-08-06 16:20:32 on dist-test-slave-t3q3"
server_key: "2048e5e336187b2be8e82d3efd0ff589"
server_key_iv: "ee445ec19ef5213a2c151fc88676087a"
server_key_version: "encryptionkey@0"
I20260806 16:20:32.013680 2648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 16:20:32.017433 2673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:32.019176 2648 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:20:32.019368 2648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "175ae1a9d17341c09827d6d11d712e75"
format_stamp: "Formatted at 2026-08-06 16:20:32 on dist-test-slave-t3q3"
server_key: "2048e5e336187b2be8e82d3efd0ff589"
server_key_iv: "ee445ec19ef5213a2c151fc88676087a"
server_key_version: "encryptionkey@0"
I20260806 16:20:32.019539 2648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:32.036959 2648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:32.041316 2648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:32.041594 2648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:32.042521 2648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:20:32.044023 2648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:20:32.044121 2648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 16:20:32.044205 2648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:20:32.044239 2648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:32.056963 2648 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:40507
I20260806 16:20:32.057015 2826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:40507 every 8 connection(s)
I20260806 16:20:32.058305 2648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:20:32.067740 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 2648
I20260806 16:20:32.068054 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:20:32.068498 12246 external_mini_cluster.cc:1428] Setting key 0a62cfc91c325101c2c20714d725dfa3
Aug 06 16:20:32 dist-test-slave-t3q3 krb5kdc[2109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033231, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:32.078284 2226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:32.061206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:56095 (local address 127.11.245.190:46857)
0806 16:20:32.061441 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:32.061445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:32.062693 (+ 1248us) server_negotiation.cc:407] Connection header received
0806 16:20:32.064752 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:32.064757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:32.064841 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:32.064965 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:32.066628 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.067233 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.068768 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.069073 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.069214 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:32.073156 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:32.073168 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:32.073171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:32.073216 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:32.075212 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:32.075957 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:32.075961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:32.075963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:32.076118 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:32.076755 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:32.076758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:32.076759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:32.077032 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:32.077131 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:32.077926 (+ 795us) server_negotiation.cc:299] Negotiation successful
0806 16:20:32.078111 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260806 16:20:32.079391 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:32.061697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:56095)
0806 16:20:32.062477 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:32.062549 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:32.064418 (+ 1869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:32.065226 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:32.065235 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:32.065759 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:32.066430 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:32.066446 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.067394 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.067398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:32.068607 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:32.068622 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.069489 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.069494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:32.069762 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:32.070608 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:32.070629 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:32.072922 (+ 2293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:32.075399 (+ 2477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:32.075407 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:32.075431 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:32.075819 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:32.076382 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:32.076389 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:32.076392 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:32.076611 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:32.077171 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:32.077179 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:32.077595 (+ 416us) client_negotiation.cc:769] Sending connection context
0806 16:20:32.077813 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 16:20:32.078211 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":158,"threads_started":1}
I20260806 16:20:32.081341 2828 heartbeater.cc:347] Connected to a master server at 127.11.245.190:46857
I20260806 16:20:32.081799 2828 heartbeater.cc:464] Registering TS with master...
I20260806 16:20:32.083178 2828 heartbeater.cc:511] Master 127.11.245.190:46857 requested a full tablet report, sending...
I20260806 16:20:32.084954 2177 ts_manager.cc:194] Registered new tserver with Master: 175ae1a9d17341c09827d6d11d712e75 (127.11.245.131:40507)
I20260806 16:20:32.085662 2177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:56095
I20260806 16:20:32.097784 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:20:32.110555 2226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:32.100784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33626 (local address 127.11.245.190:46857)
0806 16:20:32.100977 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:32.100984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:32.101177 (+ 193us) server_negotiation.cc:407] Connection header received
0806 16:20:32.101268 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:32.101271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:32.101362 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:32.101444 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:32.102446 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.103043 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.104153 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.104485 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.104596 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:32.106522 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:32.106525 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:32.106527 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:32.106559 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:32.108357 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:32.108853 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:32.108856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:32.108858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:32.108944 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:32.109302 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:32.109306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:32.109307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:32.109617 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:32.109742 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:32.110082 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 16:20:32.110277 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 16:20:32.115125 2177 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:20:32.119092 2177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:20:32.140408 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:32.133543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:40507 (local address 127.0.0.1:48892)
0806 16:20:32.133896 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:32.133912 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:32.134343 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:32.135313 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:32.135317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:32.135340 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:32.135457 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:32.135463 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.137008 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.137012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:32.138138 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:32.138150 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.139574 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.139578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:32.139728 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:32.139783 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 16:20:32.139936 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 16:20:32.140197 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":100,"thread_start_us":123,"threads_started":1}
I20260806 16:20:32.140575 2855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:32.133800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48892 (local address 127.11.245.131:40507)
0806 16:20:32.134498 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:32.134514 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:32.134553 (+ 39us) server_negotiation.cc:407] Connection header received
0806 16:20:32.134654 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:32.134661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:32.134922 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:32.135129 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:32.135604 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.136844 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.138293 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.139162 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.139302 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:32.139507 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:32.140208 (+ 701us) server_negotiation.cc:299] Negotiation successful
0806 16:20:32.140345 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":267,"threads_started":1}
I20260806 16:20:32.141036 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:32.131916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:34733 (local address 127.0.0.1:34864)
0806 16:20:32.132830 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:32.132847 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:32.133049 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:32.134189 (+ 1140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:32.134196 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:32.134228 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:32.134505 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:32.134520 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.136400 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.136420 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:32.138747 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:32.138754 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.140286 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.140290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:32.140430 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:32.140698 (+ 268us) client_negotiation.cc:769] Sending connection context
0806 16:20:32.140761 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 16:20:32.140873 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"mutex_wait_us":574,"thread_start_us":166,"threads_started":1}
I20260806 16:20:32.141455 2850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:32.131942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36776 (local address 127.11.245.130:44749)
0806 16:20:32.133187 (+ 1245us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:32.133201 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:32.133235 (+ 34us) server_negotiation.cc:407] Connection header received
0806 16:20:32.133347 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:32.133353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:32.133626 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:32.133808 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:32.134913 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.136276 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.138920 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.139872 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.140033 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:32.140208 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:32.141129 (+ 921us) server_negotiation.cc:299] Negotiation successful
0806 16:20:32.141279 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1107,"thread_start_us":242,"threads_started":1}
I20260806 16:20:32.141594 2851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:32.132256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34864 (local address 127.11.245.129:34733)
0806 16:20:32.132714 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:32.132723 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:32.133234 (+ 511us) server_negotiation.cc:407] Connection header received
0806 16:20:32.133359 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:32.133365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:32.133616 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:32.133797 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:32.134750 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.136253 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.138923 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.139875 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.140022 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:32.140208 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:32.141259 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0806 16:20:32.141394 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":144,"threads_started":1}
I20260806 16:20:32.141790 2847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:32.131734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:44749 (local address 127.0.0.1:36776)
0806 16:20:32.132769 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:32.132838 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:32.133050 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:32.134382 (+ 1332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:32.134389 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:32.134412 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:32.134715 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:32.134726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.136452 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.136454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:32.138717 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:32.138734 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.140286 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.140290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:32.140432 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:32.140507 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 16:20:32.140910 (+ 403us) client_negotiation.cc:240] Negotiation successful
0806 16:20:32.140988 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"mutex_wait_us":431,"thread_start_us":191,"threads_started":1}
I20260806 16:20:32.145109 2348 tablet_service.cc:1467] Processing CreateTablet for tablet 54ff23f1259c4c28950eeea2bfcd3dbf (DEFAULT_TABLE table=test-table [id=c4ba69c90a524b48ae2aaa447b1635ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:20:32.145119 2740 tablet_service.cc:1467] Processing CreateTablet for tablet 54ff23f1259c4c28950eeea2bfcd3dbf (DEFAULT_TABLE table=test-table [id=c4ba69c90a524b48ae2aaa447b1635ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:20:32.145119 2545 tablet_service.cc:1467] Processing CreateTablet for tablet 54ff23f1259c4c28950eeea2bfcd3dbf (DEFAULT_TABLE table=test-table [id=c4ba69c90a524b48ae2aaa447b1635ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:20:32.146539 2348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54ff23f1259c4c28950eeea2bfcd3dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:32.146541 2545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54ff23f1259c4c28950eeea2bfcd3dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:32.146551 2740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54ff23f1259c4c28950eeea2bfcd3dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:32.157091 2863 tablet_bootstrap.cc:492] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: Bootstrap starting.
I20260806 16:20:32.157282 2856 tablet_bootstrap.cc:492] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394: Bootstrap starting.
I20260806 16:20:32.159857 2863 tablet_bootstrap.cc:654] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:32.160411 2856 tablet_bootstrap.cc:654] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:32.160989 2863 log.cc:824] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:32.161221 2856 log.cc:824] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:32.162132 2860 tablet_bootstrap.cc:492] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75: Bootstrap starting.
I20260806 16:20:32.163823 2856 tablet_bootstrap.cc:492] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394: No bootstrap required, opened a new log
I20260806 16:20:32.164207 2856 ts_tablet_manager.cc:1397] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260806 16:20:32.164441 2863 tablet_bootstrap.cc:492] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: No bootstrap required, opened a new log
I20260806 16:20:32.164667 2863 ts_tablet_manager.cc:1397] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 16:20:32.165427 2860 tablet_bootstrap.cc:654] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:32.167258 2860 log.cc:824] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:32.168819 2856 raft_consensus.cc:348] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } }
I20260806 16:20:32.169206 2856 raft_consensus.cc:374] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:32.169310 2856 raft_consensus.cc:729] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51943d5d2bb04e778a7b24de28590394, State: Initialized, Role: FOLLOWER
I20260806 16:20:32.170006 2856 consensus_queue.cc:260] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } }
I20260806 16:20:32.169955 2863 raft_consensus.cc:348] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } }
I20260806 16:20:32.170305 2863 raft_consensus.cc:374] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:32.170356 2860 tablet_bootstrap.cc:492] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75: No bootstrap required, opened a new log
I20260806 16:20:32.170383 2863 raft_consensus.cc:729] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 732296a961b14967a5e2a6505fba1ba5, State: Initialized, Role: FOLLOWER
I20260806 16:20:32.170611 2860 ts_tablet_manager.cc:1397] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 16:20:32.170915 2863 consensus_queue.cc:260] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } }
I20260806 16:20:32.171439 2856 ts_tablet_manager.cc:1428] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 16:20:32.171873 2449 heartbeater.cc:503] Master 127.11.245.190:46857 was elected leader, sending a full tablet report...
I20260806 16:20:32.171880 2641 heartbeater.cc:503] Master 127.11.245.190:46857 was elected leader, sending a full tablet report...
I20260806 16:20:32.172045 2863 ts_tablet_manager.cc:1428] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260806 16:20:32.178594 2450 tablet.cc:2369] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:20:32.178242 2860 raft_consensus.cc:348] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } }
I20260806 16:20:32.178725 2860 raft_consensus.cc:374] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:32.178782 2860 raft_consensus.cc:729] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 175ae1a9d17341c09827d6d11d712e75, State: Initialized, Role: FOLLOWER
I20260806 16:20:32.179450 2860 consensus_queue.cc:260] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } }
I20260806 16:20:32.180552 2828 heartbeater.cc:503] Master 127.11.245.190:46857 was elected leader, sending a full tablet report...
I20260806 16:20:32.180995 2860 ts_tablet_manager.cc:1428] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
W20260806 16:20:32.247834 2642 tablet.cc:2369] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:20:32.309792 2830 tablet.cc:2369] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:20:32.342553 2871 raft_consensus.cc:482] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:20:32.342996 2871 raft_consensus.cc:504] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } }
I20260806 16:20:32.344771 2871 leader_election.cc:288] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51943d5d2bb04e778a7b24de28590394 (127.11.245.130:44749), 732296a961b14967a5e2a6505fba1ba5 (127.11.245.129:34733)
I20260806 16:20:32.349197 2850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:32.345423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:57595 (local address 127.11.245.130:44749)
0806 16:20:32.345594 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:32.345600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:32.345624 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:20:32.345778 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:32.345782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:32.345863 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:32.345982 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:32.346302 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.346862 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.348117 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.348542 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.348687 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:32.348763 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:32.348953 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 16:20:32.349042 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 16:20:32.349197 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:32.345233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:44749 (local address 127.11.245.131:57595)
0806 16:20:32.345444 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:32.345465 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:32.345618 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:32.345999 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:32.346002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:32.346026 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:32.346176 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:32.346182 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.346981 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.346986 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:32.347948 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:32.347960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.348687 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.348691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:32.348806 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:32.348830 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 16:20:32.348934 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 16:20:32.349025 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":48}
I20260806 16:20:32.349439 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:32.345233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:34733 (local address 127.11.245.131:39321)
0806 16:20:32.345959 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:32.345970 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:32.346093 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:32.346384 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:32.346387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:32.346403 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:32.346566 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:32.346572 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.347322 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.347327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:32.348247 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:32.348258 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.348975 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.348980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:32.349095 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:32.349123 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 16:20:32.349214 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 16:20:32.349289 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"mutex_wait_us":286,"thread_start_us":180,"threads_started":1}
I20260806 16:20:32.349445 2851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:32.345447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:39321 (local address 127.11.245.129:34733)
0806 16:20:32.345612 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:32.345618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:32.346060 (+ 442us) server_negotiation.cc:407] Connection header received
0806 16:20:32.346213 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:32.346217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:32.346281 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:32.346372 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:32.346681 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.347204 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.348374 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.348836 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.348981 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:32.349057 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:32.349234 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 16:20:32.349322 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 16:20:32.350229 2580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54ff23f1259c4c28950eeea2bfcd3dbf" candidate_uuid: "175ae1a9d17341c09827d6d11d712e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51943d5d2bb04e778a7b24de28590394" is_pre_election: true
I20260806 16:20:32.350234 2380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54ff23f1259c4c28950eeea2bfcd3dbf" candidate_uuid: "175ae1a9d17341c09827d6d11d712e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "732296a961b14967a5e2a6505fba1ba5" is_pre_election: true
I20260806 16:20:32.350656 2380 raft_consensus.cc:2454] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 175ae1a9d17341c09827d6d11d712e75 in term 0.
I20260806 16:20:32.350656 2580 raft_consensus.cc:2454] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 175ae1a9d17341c09827d6d11d712e75 in term 0.
I20260806 16:20:32.351418 2675 leader_election.cc:302] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 175ae1a9d17341c09827d6d11d712e75, 51943d5d2bb04e778a7b24de28590394; no voters:
I20260806 16:20:32.351820 2871 raft_consensus.cc:2781] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:20:32.351964 2871 raft_consensus.cc:482] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:20:32.352039 2871 raft_consensus.cc:3037] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:32.353500 2871 raft_consensus.cc:504] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } }
I20260806 16:20:32.353972 2871 leader_election.cc:288] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [CANDIDATE]: Term 1 election: Requested vote from peers 51943d5d2bb04e778a7b24de28590394 (127.11.245.130:44749), 732296a961b14967a5e2a6505fba1ba5 (127.11.245.129:34733)
I20260806 16:20:32.354561 2380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54ff23f1259c4c28950eeea2bfcd3dbf" candidate_uuid: "175ae1a9d17341c09827d6d11d712e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "732296a961b14967a5e2a6505fba1ba5"
I20260806 16:20:32.354717 2380 raft_consensus.cc:3037] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:32.354781 2580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54ff23f1259c4c28950eeea2bfcd3dbf" candidate_uuid: "175ae1a9d17341c09827d6d11d712e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51943d5d2bb04e778a7b24de28590394"
I20260806 16:20:32.355213 2580 raft_consensus.cc:3037] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:32.356336 2380 raft_consensus.cc:2454] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 175ae1a9d17341c09827d6d11d712e75 in term 1.
I20260806 16:20:32.356798 2676 leader_election.cc:302] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 175ae1a9d17341c09827d6d11d712e75, 732296a961b14967a5e2a6505fba1ba5; no voters:
I20260806 16:20:32.357012 2871 raft_consensus.cc:2781] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:20:32.357348 2871 raft_consensus.cc:686] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [term 1 LEADER]: Becoming Leader. State: Replica: 175ae1a9d17341c09827d6d11d712e75, State: Running, Role: LEADER
I20260806 16:20:32.357831 2871 consensus_queue.cc:237] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } }
I20260806 16:20:32.358117 2580 raft_consensus.cc:2454] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 175ae1a9d17341c09827d6d11d712e75 in term 1.
I20260806 16:20:32.362726 2177 catalog_manager.cc:5495] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 175ae1a9d17341c09827d6d11d712e75 (127.11.245.131). New cstate: current_term: 1 leader_uuid: "175ae1a9d17341c09827d6d11d712e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } health_report { overall_health: HEALTHY } } }
I20260806 16:20:32.384912 2855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:32.380407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48904 (local address 127.11.245.131:40507)
0806 16:20:32.380572 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:32.380576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:32.380592 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:20:32.380704 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:32.380707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:32.380775 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:32.380896 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:20:32.381197 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.381800 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.382783 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:32.383031 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:32.383124 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:32.383568 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:20:32.384332 (+ 764us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:20:32.384457 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:32.384568 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 16:20:32.384641 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 16:20:32.393226 2380 raft_consensus.cc:1261] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Refusing update from remote peer 175ae1a9d17341c09827d6d11d712e75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:20:32.393316 2580 raft_consensus.cc:1261] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394 [term 1 FOLLOWER]: Refusing update from remote peer 175ae1a9d17341c09827d6d11d712e75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:20:32.394522 2874 consensus_queue.cc:1035] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:20:32.394814 2871 consensus_queue.cc:1035] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:20:32.405519 2879 mvcc.cc:205] Tried to move back new op lower bound from 7315592119872774144 to 7315592119741902848. Current Snapshot: MvccSnapshot[applied={T|T < 7315592119872774144}]
I20260806 16:20:32.407356 2882 mvcc.cc:205] Tried to move back new op lower bound from 7315592119872774144 to 7315592119741902848. Current Snapshot: MvccSnapshot[applied={T|T < 7315592119872774144}]
I20260806 16:20:32.407611 2881 mvcc.cc:205] Tried to move back new op lower bound from 7315592119872774144 to 7315592119741902848. Current Snapshot: MvccSnapshot[applied={T|T < 7315592119872774144}]
I20260806 16:20:32.418551 2177 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:20:32.418853 2177 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:20:32.422492 2177 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Sending DeleteTablet for 3 replicas of tablet 54ff23f1259c4c28950eeea2bfcd3dbf
I20260806 16:20:32.423731 2740 tablet_service.cc:1514] Processing DeleteTablet for tablet 54ff23f1259c4c28950eeea2bfcd3dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:20:32 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:48892
I20260806 16:20:32.423731 2545 tablet_service.cc:1514] Processing DeleteTablet for tablet 54ff23f1259c4c28950eeea2bfcd3dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:20:32 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:36776
I20260806 16:20:32.423895 2348 tablet_service.cc:1514] Processing DeleteTablet for tablet 54ff23f1259c4c28950eeea2bfcd3dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:20:32 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:34864
I20260806 16:20:32.424533 2896 tablet_replica.cc:330] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394: stopping tablet replica
I20260806 16:20:32.424811 2897 tablet_replica.cc:330] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75: stopping tablet replica
I20260806 16:20:32.424964 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 2260
I20260806 16:20:32.444481 2897 raft_consensus.cc:2229] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:20:32.445538 2897 raft_consensus.cc:2258] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:20:32.446202 2896 raft_consensus.cc:2229] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:20:32.446729 2896 raft_consensus.cc:2258] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:20:32.448349 2897 ts_tablet_manager.cc:1905] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:20:32.448383 2896 ts_tablet_manager.cc:1905] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:20:32.451937 2896 ts_tablet_manager.cc:1918] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:20:32.451937 2897 ts_tablet_manager.cc:1918] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:20:32.452152 2896 log.cc:1196] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/54ff23f1259c4c28950eeea2bfcd3dbf
I20260806 16:20:32.452201 2897 log.cc:1196] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/54ff23f1259c4c28950eeea2bfcd3dbf
I20260806 16:20:32.452593 2896 ts_tablet_manager.cc:1939] T 54ff23f1259c4c28950eeea2bfcd3dbf P 51943d5d2bb04e778a7b24de28590394: Deleting consensus metadata
I20260806 16:20:32.452781 2897 ts_tablet_manager.cc:1939] T 54ff23f1259c4c28950eeea2bfcd3dbf P 175ae1a9d17341c09827d6d11d712e75: Deleting consensus metadata
I20260806 16:20:32.454480 2157 catalog_manager.cc:4841] TS 175ae1a9d17341c09827d6d11d712e75 (127.11.245.131:40507): tablet 54ff23f1259c4c28950eeea2bfcd3dbf (table test-table [id=c4ba69c90a524b48ae2aaa447b1635ef]) successfully deleted
I20260806 16:20:32.454480 2156 catalog_manager.cc:4841] TS 51943d5d2bb04e778a7b24de28590394 (127.11.245.130:44749): tablet 54ff23f1259c4c28950eeea2bfcd3dbf (table test-table [id=c4ba69c90a524b48ae2aaa447b1635ef]) successfully deleted
W20260806 16:20:32.457160 2157 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:34733 (error 108)
I20260806 16:20:32.457496 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 2456
I20260806 16:20:32.466105 2847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:32.465614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:34733 (local address 127.0.0.1:34874)
0806 16:20:32.465883 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:32.465970 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:34733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
W20260806 16:20:32.466558 2157 catalog_manager.cc:4584] TS 732296a961b14967a5e2a6505fba1ba5 (127.11.245.129:34733): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 54ff23f1259c4c28950eeea2bfcd3dbf: Network error: Client connection negotiation failed: client connection to 127.11.245.129:34733: connect: Connection refused (error 111)
I20260806 16:20:32.478595 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 2648
I20260806 16:20:32.499689 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 2127
I20260806 16:20:52.519820 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:46857
--webserver_interface=127.11.245.190
--webserver_port=36501
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:46857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:20:52.675886 2901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:52.676644 2901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:52.676760 2901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:52.681456 2901 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 16:20:52.681576 2901 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 16:20:52.681607 2901 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 16:20:52.681629 2901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:20:52.681679 2901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:52.681738 2901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:52.681778 2901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:20:52.681819 2901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:20:52.689110 2901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:46857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:46857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=36501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:52.690802 2901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:52.692263 2901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:52.711928 2913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:52.711928 2910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:52.712023 2901 server_base.cc:1034] running on GCE node
W20260806 16:20:52.711928 2911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:52.713620 2901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:52.717238 2901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:52.718528 2901 hybrid_clock.cc:648] HybridClock initialized: now 1786033252718509 us; error 54 us; skew 500 ppm
Aug 06 16:20:52 dist-test-slave-t3q3 krb5kdc[2109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033252, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:52.722692 2901 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:20:52.724336 2901 webserver.cc:460] Webserver started at http://127.11.245.190:36501/ using document root <none> and password file <none>
I20260806 16:20:52.725215 2901 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:52.725323 2901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:52.732350 2901 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:20:52.736382 2926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:52.738734 2901 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:20:52.738948 2901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "36797ff4d85440b78deada47e23ef7ce"
format_stamp: "Formatted at 2026-08-06 16:20:30 on dist-test-slave-t3q3"
server_key: "d8c2ff7a3f2f698df5ecf8b6ef9399de"
server_key_iv: "cbb0558cd6f955ae4eaf004917ead08a"
server_key_version: "encryptionkey@0"
I20260806 16:20:52.740036 2901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:52.755023 2901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:52.761281 2901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:52.761597 2901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:52.771118 2901 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:46857
I20260806 16:20:52.771473 3000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:46857 every 8 connection(s)
I20260806 16:20:52.772856 2901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:20:52.779645 3001 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:20:52.782120 3001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Bootstrap starting.
I20260806 16:20:52.782781 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 2901
I20260806 16:20:52.783442 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:34733
--local_ip_for_outbound_sockets=127.11.245.129
--tserver_master_addrs=127.11.245.190:46857
--webserver_port=41653
--webserver_interface=127.11.245.129
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:20:52.809523 3001 log.cc:824] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:52.822687 3001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:20:52.823285 3001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Bootstrap complete.
I20260806 16:20:52.829114 3001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } }
I20260806 16:20:52.829668 3001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36797ff4d85440b78deada47e23ef7ce, State: Initialized, Role: FOLLOWER
I20260806 16:20:52.830358 3001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } }
I20260806 16:20:52.830506 3001 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:20:52.830615 3001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:20:52.830749 3001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 1 FOLLOWER]: Advancing to term 2
I20260806 16:20:52.833520 3001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } }
I20260806 16:20:52.834004 3001 leader_election.cc:302] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36797ff4d85440b78deada47e23ef7ce; no voters:
I20260806 16:20:52.834509 3001 leader_election.cc:288] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 16:20:52.834640 3006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 2 FOLLOWER]: Leader election won for term 2
I20260806 16:20:52.834882 3006 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 2 LEADER]: Becoming Leader. State: Replica: 36797ff4d85440b78deada47e23ef7ce, State: Running, Role: LEADER
I20260806 16:20:52.835336 3006 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } }
I20260806 16:20:52.836292 3001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:20:52.837539 3007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "36797ff4d85440b78deada47e23ef7ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } } }
I20260806 16:20:52.837716 3007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [sys.catalog]: This master's current role is: LEADER
I20260806 16:20:52.838418 3008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36797ff4d85440b78deada47e23ef7ce. Latest consensus state: current_term: 2 leader_uuid: "36797ff4d85440b78deada47e23ef7ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } } }
I20260806 16:20:52.838565 3008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [sys.catalog]: This master's current role is: LEADER
I20260806 16:20:52.842481 3018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:20:52.850535 3018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:20:52.851763 3018 catalog_manager.cc:1259] Loaded cluster ID: 3bacf4970a3c45c88a923244fd5637e3
I20260806 16:20:52.851935 3018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:20:52.855104 3018 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:20:52.856253 3018 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Loaded TSK: 0
I20260806 16:20:52.872748 3018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Generated new TSK 1
I20260806 16:20:52.873330 3018 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 16:20:52.952281 3004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:52.952684 3004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:52.952776 3004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:52.957265 3004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:52.957376 3004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:52.957479 3004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:20:52.964195 3004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:34733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=41653
--tserver_master_addrs=127.11.245.190:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:52.965720 3004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:52.967082 3004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:52.981846 3043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:52.981777 3042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:52.981796 3046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:52.982556 3004 server_base.cc:1034] running on GCE node
I20260806 16:20:52.983238 3004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:52.984510 3004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:52.985733 3004 hybrid_clock.cc:648] HybridClock initialized: now 1786033252985683 us; error 65 us; skew 500 ppm
Aug 06 16:20:52 dist-test-slave-t3q3 krb5kdc[2109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033252, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:52.990471 3004 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:20:52.992223 3004 webserver.cc:460] Webserver started at http://127.11.245.129:41653/ using document root <none> and password file <none>
I20260806 16:20:52.993160 3004 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:52.993278 3004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:53.000128 3004 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:20:53.004293 3057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:53.006711 3004 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:20:53.007056 3004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "732296a961b14967a5e2a6505fba1ba5"
format_stamp: "Formatted at 2026-08-06 16:20:31 on dist-test-slave-t3q3"
server_key: "d3f588f0a1b8d17aba4282ba855a87ab"
server_key_iv: "cbd0af4711cd3e570674beabbf879ce9"
server_key_version: "encryptionkey@0"
I20260806 16:20:53.008174 3004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:53.027199 3004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:53.031513 3004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:53.031728 3004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:53.032528 3004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:20:53.033915 3066 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 16:20:53.040442 3004 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 16:20:53.040539 3004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260806 16:20:53.040644 3004 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 16:20:53.043032 3004 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 16:20:53.043092 3004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 16:20:53.043602 3066 tablet_bootstrap.cc:492] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: Bootstrap starting.
I20260806 16:20:53.057673 3004 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:34733
I20260806 16:20:53.057785 3213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:34733 every 8 connection(s)
I20260806 16:20:53.059073 3004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:20:53.066346 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 3004
I20260806 16:20:53.067194 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:44749
--local_ip_for_outbound_sockets=127.11.245.130
--tserver_master_addrs=127.11.245.190:46857
--webserver_port=45981
--webserver_interface=127.11.245.130
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:20:53.071770 3066 log.cc:824] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:20:53 dist-test-slave-t3q3 krb5kdc[2109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033252, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:53.083196 3066 tablet_bootstrap.cc:492] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:20:53.083897 3066 tablet_bootstrap.cc:492] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: Bootstrap complete.
I20260806 16:20:53.084940 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.062327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:57309 (local address 127.11.245.190:46857)
0806 16:20:53.062928 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:53.062968 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:53.063545 (+ 577us) server_negotiation.cc:407] Connection header received
0806 16:20:53.065677 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:53.065692 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:53.066189 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:53.066673 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:53.068488 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.070297 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.072189 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.073140 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.073392 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.077321 (+ 3929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:53.077328 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:53.077348 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:53.077393 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:53.080054 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:53.081058 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:53.081065 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:53.081071 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:53.081259 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:53.081764 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:53.081767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:53.081769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:53.082370 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:53.082518 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:53.083453 (+ 935us) server_negotiation.cc:299] Negotiation successful
0806 16:20:53.083844 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":205,"threads_started":1}
I20260806 16:20:53.085139 3219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.062331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:57309)
0806 16:20:53.063243 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:53.063359 (+ 116us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:53.064981 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:53.066745 (+ 1764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:53.066755 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:53.067418 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:53.068152 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:53.068191 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.070521 (+ 2330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.070526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:53.072008 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:53.072025 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.073331 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.073336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:53.073537 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.074400 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:53.074425 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:53.076993 (+ 2568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:53.080314 (+ 3321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:53.080326 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:53.080348 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:53.080881 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:53.081390 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:53.081396 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:53.081401 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:53.081633 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:53.082516 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:53.082528 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:53.082936 (+ 408us) client_negotiation.cc:769] Sending connection context
0806 16:20:53.083199 (+ 263us) client_negotiation.cc:240] Negotiation successful
0806 16:20:53.083724 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":197,"threads_started":1}
I20260806 16:20:53.085860 3066 ts_tablet_manager.cc:1397] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: Time spent bootstrapping tablet: real 0.043s user 0.033s sys 0.007s
I20260806 16:20:53.088395 3214 heartbeater.cc:347] Connected to a master server at 127.11.245.190:46857
I20260806 16:20:53.088701 3214 heartbeater.cc:464] Registering TS with master...
I20260806 16:20:53.089450 3214 heartbeater.cc:511] Master 127.11.245.190:46857 requested a full tablet report, sending...
I20260806 16:20:53.089264 3066 raft_consensus.cc:348] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } }
I20260806 16:20:53.090005 3066 raft_consensus.cc:729] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 732296a961b14967a5e2a6505fba1ba5, State: Initialized, Role: FOLLOWER
I20260806 16:20:53.090663 3066 consensus_queue.cc:260] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } }
I20260806 16:20:53.092186 3066 ts_tablet_manager.cc:1428] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 16:20:53.095013 2943 ts_manager.cc:194] Registered new tserver with Master: 732296a961b14967a5e2a6505fba1ba5 (127.11.245.129:34733)
I20260806 16:20:53.101454 2943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:57309
I20260806 16:20:53.104911 3214 heartbeater.cc:503] Master 127.11.245.190:46857 was elected leader, sending a full tablet report...
Aug 06 16:20:53 dist-test-slave-t3q3 krb5kdc[2109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033252, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for kudu/127.11.245.129@KRBTEST.COM
I20260806 16:20:53.115374 3227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.100255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:34733 (local address 127.0.0.1:50804)
0806 16:20:53.101228 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:53.101281 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:53.101551 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:53.102325 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:53.102330 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:53.102841 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:53.102977 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:53.102985 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.104575 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.104579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:53.105496 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:53.105508 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.106660 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.106664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:53.106802 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.107765 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:53.107783 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:53.109855 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:53.112801 (+ 2946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:53.112822 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:53.112830 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:53.113167 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:53.113606 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:53.113612 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:53.113613 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:53.113715 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:53.114445 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:53.114453 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:53.114681 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 16:20:53.114843 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 16:20:53.115145 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":322,"threads_started":1}
I20260806 16:20:53.115448 3226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.100295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50804 (local address 127.11.245.129:34733)
0806 16:20:53.100879 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:53.100894 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:53.101412 (+ 518us) server_negotiation.cc:407] Connection header received
0806 16:20:53.101784 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:53.101790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:53.102007 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:53.102115 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:53.103114 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.104425 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.105684 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.106254 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.106458 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.110062 (+ 3604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:53.110079 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:53.110097 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:53.110133 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:53.112587 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:53.113311 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:53.113320 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:53.113322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:53.113460 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:53.113830 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:53.113834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:53.113836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:53.114272 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:53.114435 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:53.114983 (+ 548us) server_negotiation.cc:299] Negotiation successful
0806 16:20:53.115222 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":271,"threads_started":1}
I20260806 16:20:53.117461 3124 tablet_service.cc:1514] Processing DeleteTablet for tablet 54ff23f1259c4c28950eeea2bfcd3dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:20:32 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:50804
I20260806 16:20:53.118865 3228 tablet_replica.cc:330] stopping tablet replica
I20260806 16:20:53.119326 3228 raft_consensus.cc:2229] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:20:53.119690 3228 raft_consensus.cc:2258] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:20:53.121177 3228 ts_tablet_manager.cc:1905] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:20:53.126125 3228 ts_tablet_manager.cc:1918] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:20:53.126235 3228 log.cc:1196] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal/wals/54ff23f1259c4c28950eeea2bfcd3dbf
I20260806 16:20:53.126578 3228 ts_tablet_manager.cc:1939] T 54ff23f1259c4c28950eeea2bfcd3dbf P 732296a961b14967a5e2a6505fba1ba5: Deleting consensus metadata
I20260806 16:20:53.128969 2929 catalog_manager.cc:4841] TS 732296a961b14967a5e2a6505fba1ba5 (127.11.245.129:34733): tablet 54ff23f1259c4c28950eeea2bfcd3dbf (table test-table [id=c4ba69c90a524b48ae2aaa447b1635ef]) successfully deleted
W20260806 16:20:53.232333 3220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:53.232685 3220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:53.232735 3220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:53.236999 3220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:53.237097 3220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:53.237160 3220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:20:53.243093 3220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:44749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=45981
--tserver_master_addrs=127.11.245.190:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:53.244642 3220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:53.246076 3220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:53.259128 3234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:53.259155 3235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:53.259214 3220 server_base.cc:1034] running on GCE node
W20260806 16:20:53.259146 3238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:53.260035 3220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:53.260936 3220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:53.262144 3220 hybrid_clock.cc:648] HybridClock initialized: now 1786033253262125 us; error 34 us; skew 500 ppm
Aug 06 16:20:53 dist-test-slave-t3q3 krb5kdc[2109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033253, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:53.265676 3220 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:20:53.266860 3220 webserver.cc:460] Webserver started at http://127.11.245.130:45981/ using document root <none> and password file <none>
I20260806 16:20:53.267627 3220 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:53.267760 3220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:53.273030 3220 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:20:53.275717 3247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:53.277453 3220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 16:20:53.277644 3220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "51943d5d2bb04e778a7b24de28590394"
format_stamp: "Formatted at 2026-08-06 16:20:31 on dist-test-slave-t3q3"
server_key: "82da88dc13669f9347fb54acfec7b349"
server_key_iv: "1a32a126b8660a51ed8817a1a174c4dc"
server_key_version: "encryptionkey@0"
I20260806 16:20:53.278431 3220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:53.299573 3220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:53.303769 3220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:53.304105 3220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:53.305011 3220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:20:53.306137 3220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:20:53.306217 3220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:53.306298 3220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:20:53.306345 3220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:53.317919 3220 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:44749
I20260806 16:20:53.317939 3408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:44749 every 8 connection(s)
I20260806 16:20:53.319350 3220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:20:53.319960 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 3220
I20260806 16:20:53.320529 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:40507
--local_ip_for_outbound_sockets=127.11.245.131
--tserver_master_addrs=127.11.245.190:46857
--webserver_port=33937
--webserver_interface=127.11.245.131
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:20:53 dist-test-slave-t3q3 krb5kdc[2109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033253, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:53.338397 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.322354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:45877 (local address 127.11.245.190:46857)
0806 16:20:53.322565 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:53.322571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:53.323968 (+ 1397us) server_negotiation.cc:407] Connection header received
0806 16:20:53.325911 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:53.325916 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:53.325997 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:53.326118 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:53.327830 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.328544 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.329969 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.330252 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.330384 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.333875 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:53.333879 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:53.333882 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:53.333915 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:53.335623 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:53.336313 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:53.336317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:53.336319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:53.336438 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:53.336913 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:53.336917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:53.336919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:53.337221 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:53.337343 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:53.338003 (+ 660us) server_negotiation.cc:299] Negotiation successful
0806 16:20:53.338174 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 16:20:53.339617 3416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.322997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:45877)
0806 16:20:53.323758 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:53.323820 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:53.325624 (+ 1804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:53.326377 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:53.326387 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:53.326859 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:53.327632 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:53.327650 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.328709 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.328713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:53.329813 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:53.329825 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.330454 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.330459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:53.330667 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.331432 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:53.331453 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:53.333650 (+ 2197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:53.335767 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:53.335775 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:53.335798 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:53.336201 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:53.336543 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:53.336547 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:53.336549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:53.336795 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:53.337341 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:53.337350 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:53.337689 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 16:20:53.337944 (+ 255us) client_negotiation.cc:240] Negotiation successful
0806 16:20:53.338377 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":212,"threads_started":1}
I20260806 16:20:53.341372 3410 heartbeater.cc:347] Connected to a master server at 127.11.245.190:46857
I20260806 16:20:53.341755 3410 heartbeater.cc:464] Registering TS with master...
I20260806 16:20:53.342581 3410 heartbeater.cc:511] Master 127.11.245.190:46857 requested a full tablet report, sending...
I20260806 16:20:53.344143 2943 ts_manager.cc:194] Registered new tserver with Master: 51943d5d2bb04e778a7b24de28590394 (127.11.245.130:44749)
I20260806 16:20:53.345088 2943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:45877
W20260806 16:20:53.466315 3413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:20:53.466706 3413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:20:53.466755 3413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:20:53.471256 3413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:20:53.471350 3413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:20:53.471426 3413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:20:53.477394 3413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:40507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=33937
--tserver_master_addrs=127.11.245.190:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:20:53.478854 3413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:20:53.480180 3413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:20:53.492681 3428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:53.492635 3424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:20:53.492653 3423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:20:53.492659 3413 server_base.cc:1034] running on GCE node
I20260806 16:20:53.493522 3413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:20:53.494669 3413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:20:53.495870 3413 hybrid_clock.cc:648] HybridClock initialized: now 1786033253495846 us; error 35 us; skew 500 ppm
Aug 06 16:20:53 dist-test-slave-t3q3 krb5kdc[2109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033253, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:20:53.499647 3413 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:20:53.500996 3413 webserver.cc:460] Webserver started at http://127.11.245.131:33937/ using document root <none> and password file <none>
I20260806 16:20:53.501744 3413 fs_manager.cc:359] Metadata directory not provided
I20260806 16:20:53.501879 3413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:20:53.507079 3413 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 16:20:53.510092 3437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:53.511891 3413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:20:53.512136 3413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "175ae1a9d17341c09827d6d11d712e75"
format_stamp: "Formatted at 2026-08-06 16:20:32 on dist-test-slave-t3q3"
server_key: "2048e5e336187b2be8e82d3efd0ff589"
server_key_iv: "ee445ec19ef5213a2c151fc88676087a"
server_key_version: "encryptionkey@0"
I20260806 16:20:53.512993 3413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:20:53.530768 3413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:20:53.534849 3413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:20:53.535145 3413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:20:53.535949 3413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:20:53.537173 3413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:20:53.537246 3413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:53.537326 3413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:20:53.537365 3413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:20:53.549760 3413 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:40507
I20260806 16:20:53.549796 3593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:40507 every 8 connection(s)
I20260806 16:20:53.551132 3413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:20:53.551730 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 3413
I20260806 16:20:53.562889 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.552736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36156 (local address 127.11.245.190:46857)
0806 16:20:53.552932 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:53.552937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:53.553200 (+ 263us) server_negotiation.cc:407] Connection header received
0806 16:20:53.553351 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:53.553354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:53.553429 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:53.553552 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:53.554887 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.555624 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.556825 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.557218 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.557343 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.559116 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:53.559121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:53.559124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:53.559151 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:53.560945 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:53.561480 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:53.561484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:53.561486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:53.561589 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:53.561917 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:53.561921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:53.561923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:53.562159 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:53.562438 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:53.562523 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 16:20:53.562672 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 16:20:53 dist-test-slave-t3q3 krb5kdc[2109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033253, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:20:53.571161 3598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.554501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:45195 (local address 127.11.245.190:46857)
0806 16:20:53.554798 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:53.554802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:53.556116 (+ 1314us) server_negotiation.cc:407] Connection header received
0806 16:20:53.558079 (+ 1963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:53.558084 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:53.558149 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:53.558270 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:53.560066 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.560959 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.562741 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.563086 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.563196 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.566394 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:53.566398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:53.566400 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:53.566429 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:53.568450 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:53.569103 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:53.569106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:53.569108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:53.569203 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:53.569721 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:53.569724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:53.569725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:53.569952 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:53.570059 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:53.570755 (+ 696us) server_negotiation.cc:299] Negotiation successful
0806 16:20:53.570942 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":131,"threads_started":1}
I20260806 16:20:53.572422 3601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.555193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:45195)
0806 16:20:53.555926 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:53.555981 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:53.557794 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:53.558488 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:53.558496 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:53.558948 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:20:53.559818 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:53.559850 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.561138 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.561142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:53.562590 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:53.562602 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.563278 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.563285 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:53.563534 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.564247 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:20:53.564273 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:20:53.566163 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:20:53.568606 (+ 2443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:53.568613 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:53.568631 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:53.568967 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:53.569358 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:20:53.569396 (+ 38us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:20:53.569398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:20:53.569578 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:20:53.570075 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:20:53.570082 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:20:53.570439 (+ 357us) client_negotiation.cc:769] Sending connection context
0806 16:20:53.570706 (+ 267us) client_negotiation.cc:240] Negotiation successful
0806 16:20:53.571155 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":155,"threads_started":1}
I20260806 16:20:53.574164 3595 heartbeater.cc:347] Connected to a master server at 127.11.245.190:46857
I20260806 16:20:53.574504 3595 heartbeater.cc:464] Registering TS with master...
I20260806 16:20:53.575182 3595 heartbeater.cc:511] Master 127.11.245.190:46857 requested a full tablet report, sending...
I20260806 16:20:53.576570 2943 ts_manager.cc:194] Registered new tserver with Master: 175ae1a9d17341c09827d6d11d712e75 (127.11.245.131:40507)
I20260806 16:20:53.577291 2943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:45195
I20260806 16:20:53.577680 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:20:53.582542 3598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.578434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36164 (local address 127.11.245.190:46857)
0806 16:20:53.578598 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:53.578603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:53.578871 (+ 268us) server_negotiation.cc:407] Connection header received
0806 16:20:53.579076 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:53.579079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:53.579193 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:53.579287 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:20:53.579563 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.580184 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.581087 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.581386 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.581491 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.581894 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:20:53.582073 (+ 179us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 16:20:53.582330 (+ 257us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36164: token expired
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 16:20:53.582648 3598 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36164: token expired
W20260806 16:20:53.583195 2837 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.245.190:46857)
I20260806 16:20:53.592387 3598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.583687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36178 (local address 127.11.245.190:46857)
0806 16:20:53.583816 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:53.583819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:53.583832 (+ 13us) server_negotiation.cc:407] Connection header received
0806 16:20:53.584165 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:53.584169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:53.584227 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:53.584300 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:20:53.585032 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.585580 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.586458 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.586695 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.586791 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.588447 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:20:53.588451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:20:53.588453 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:20:53.588480 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:20:53.590119 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:53.590570 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:53.590608 (+ 38us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:53.590611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:53.590747 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:20:53.591081 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:20:53.591086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:20:53.591087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:20:53.591331 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:20:53.591499 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:53.591884 (+ 385us) server_negotiation.cc:299] Negotiation successful
0806 16:20:53.592029 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 16:20:53.596215 2943 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:20:53.601212 2943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:20:53.612007 3124 tablet_service.cc:1467] Processing CreateTablet for tablet d4c1767aeb1e49ab92a8d482e738d2b7 (DEFAULT_TABLE table=test-table [id=85d74d7af2f84ff48cc8d7c7aadd32cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:20:53.612996 3124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4c1767aeb1e49ab92a8d482e738d2b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:53.616842 3612 tablet_bootstrap.cc:492] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5: Bootstrap starting.
I20260806 16:20:53.617822 3227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.610460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:44749 (local address 127.0.0.1:48948)
0806 16:20:53.610616 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:53.610636 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:53.610738 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:53.612227 (+ 1489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:53.612231 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:53.612271 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:53.612724 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:53.612730 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.614438 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.614442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:53.615960 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:53.615972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.617103 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.617107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:53.617366 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.617428 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 16:20:53.617510 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 16:20:53.617634 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":545,"spinlock_wait_cycles":8832}
I20260806 16:20:53.617848 3609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.610758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48948 (local address 127.11.245.130:44749)
0806 16:20:53.611459 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:53.611476 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:53.611522 (+ 46us) server_negotiation.cc:407] Connection header received
0806 16:20:53.611649 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:53.611656 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:53.611988 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:53.612563 (+ 575us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:53.612829 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.614282 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.616129 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.616947 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.617299 (+ 352us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.617497 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:53.617562 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 16:20:53.617702 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":192,"threads_started":1}
I20260806 16:20:53.618508 3612 tablet_bootstrap.cc:654] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:53.618556 3607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.610779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:40507 (local address 127.0.0.1:40532)
0806 16:20:53.611120 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:53.611138 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:53.611265 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:53.612227 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:53.612231 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:53.612269 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:53.612507 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:53.612516 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.614521 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.614527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:53.615740 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:53.615753 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.617103 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.617107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:53.618158 (+ 1051us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.618185 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 16:20:53.618264 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 16:20:53.618341 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":953,"spinlock_wait_cycles":6912,"thread_start_us":141,"threads_started":1}
I20260806 16:20:53.618827 3610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.611155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40532 (local address 127.11.245.131:40507)
0806 16:20:53.611591 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:53.611605 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:53.611629 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:20:53.611712 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:53.611717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:53.611968 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:53.612872 (+ 904us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:53.613108 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.614413 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.615939 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.616947 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.617309 (+ 362us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.617494 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:53.618323 (+ 829us) server_negotiation.cc:299] Negotiation successful
0806 16:20:53.618447 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":174,"threads_started":1}
I20260806 16:20:53.620170 3612 tablet_bootstrap.cc:492] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5: No bootstrap required, opened a new log
I20260806 16:20:53.620280 3612 ts_tablet_manager.cc:1397] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 16:20:53.620769 3612 raft_consensus.cc:348] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } }
I20260806 16:20:53.620937 3612 raft_consensus.cc:374] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:53.620990 3612 raft_consensus.cc:729] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 732296a961b14967a5e2a6505fba1ba5, State: Initialized, Role: FOLLOWER
I20260806 16:20:53.621136 3612 consensus_queue.cc:260] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } }
I20260806 16:20:53.621531 3612 ts_tablet_manager.cc:1428] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:20:53.622184 3503 tablet_service.cc:1467] Processing CreateTablet for tablet d4c1767aeb1e49ab92a8d482e738d2b7 (DEFAULT_TABLE table=test-table [id=85d74d7af2f84ff48cc8d7c7aadd32cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:20:53.622170 3318 tablet_service.cc:1467] Processing CreateTablet for tablet d4c1767aeb1e49ab92a8d482e738d2b7 (DEFAULT_TABLE table=test-table [id=85d74d7af2f84ff48cc8d7c7aadd32cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:20:53.623533 3318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4c1767aeb1e49ab92a8d482e738d2b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:53.623539 3503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4c1767aeb1e49ab92a8d482e738d2b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:20:53.632203 3616 tablet_bootstrap.cc:492] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394: Bootstrap starting.
I20260806 16:20:53.632655 3617 tablet_bootstrap.cc:492] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75: Bootstrap starting.
I20260806 16:20:53.635272 3617 tablet_bootstrap.cc:654] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:53.635824 3616 tablet_bootstrap.cc:654] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394: Neither blocks nor log segments found. Creating new log.
I20260806 16:20:53.636880 3617 log.cc:824] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:53.636888 3616 log.cc:824] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394: Log is configured to *not* fsync() on all Append() calls
I20260806 16:20:53.640584 3617 tablet_bootstrap.cc:492] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75: No bootstrap required, opened a new log
I20260806 16:20:53.640584 3616 tablet_bootstrap.cc:492] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394: No bootstrap required, opened a new log
I20260806 16:20:53.640909 3617 ts_tablet_manager.cc:1397] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260806 16:20:53.640914 3616 ts_tablet_manager.cc:1397] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260806 16:20:53.646334 3617 raft_consensus.cc:348] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } }
I20260806 16:20:53.646334 3616 raft_consensus.cc:348] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } }
I20260806 16:20:53.646770 3616 raft_consensus.cc:374] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:53.646770 3617 raft_consensus.cc:374] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:20:53.646852 3617 raft_consensus.cc:729] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 175ae1a9d17341c09827d6d11d712e75, State: Initialized, Role: FOLLOWER
I20260806 16:20:53.646865 3616 raft_consensus.cc:729] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51943d5d2bb04e778a7b24de28590394, State: Initialized, Role: FOLLOWER
I20260806 16:20:53.647492 3616 consensus_queue.cc:260] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } }
I20260806 16:20:53.647492 3617 consensus_queue.cc:260] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } }
I20260806 16:20:53.648542 3410 heartbeater.cc:503] Master 127.11.245.190:46857 was elected leader, sending a full tablet report...
I20260806 16:20:53.648919 3617 ts_tablet_manager.cc:1428] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260806 16:20:53.648953 3616 ts_tablet_manager.cc:1428] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394: Time spent starting tablet: real 0.008s user 0.005s sys 0.003s
I20260806 16:20:53.650045 3595 heartbeater.cc:503] Master 127.11.245.190:46857 was elected leader, sending a full tablet report...
W20260806 16:20:53.802989 3596 tablet.cc:2369] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:20:53.811239 3215 tablet.cc:2369] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:20:53.821103 3412 tablet.cc:2369] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:20:53.964743 3621 raft_consensus.cc:482] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:20:53.965157 3621 raft_consensus.cc:504] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } }
I20260806 16:20:53.966918 3621 leader_election.cc:288] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 175ae1a9d17341c09827d6d11d712e75 (127.11.245.131:40507), 732296a961b14967a5e2a6505fba1ba5 (127.11.245.129:34733)
I20260806 16:20:53.972399 3610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.967500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:48037 (local address 127.11.245.131:40507)
0806 16:20:53.967705 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:53.967710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:53.967877 (+ 167us) server_negotiation.cc:407] Connection header received
0806 16:20:53.968140 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:53.968145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:53.968220 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:53.968284 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:53.968957 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.969674 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.971102 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.971633 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.971750 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.971809 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:53.972128 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 16:20:53.972237 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 16:20:53.972424 3622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.967400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:40507 (local address 127.11.245.130:48037)
0806 16:20:53.967747 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:53.967765 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:53.967928 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:53.968505 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:53.968508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:53.968530 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:53.968721 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:53.968729 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.969845 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.969850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:53.970941 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:53.970953 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.971779 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.971785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:53.971934 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.971972 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 16:20:53.972087 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 16:20:53.972193 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":132,"threads_started":1}
I20260806 16:20:53.972553 3624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.968137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:55057 (local address 127.11.245.129:34733)
0806 16:20:53.968540 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:53.968546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:53.968563 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:20:53.968630 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:53.968635 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:53.968729 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:53.968877 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:20:53.969276 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.970028 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.971227 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.971820 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.971962 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.972216 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:53.972263 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 16:20:53.972352 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":125,"threads_started":1}
I20260806 16:20:53.972457 3623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:53.967897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:34733 (local address 127.11.245.130:55057)
0806 16:20:53.968256 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:53.968277 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:20:53.968472 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:20:53.968912 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:20:53.968915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:20:53.968942 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:20:53.969096 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:53.969104 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.970156 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.970159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:53.971080 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:20:53.971092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:20:53.971964 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:20:53.971967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:20:53.972123 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:53.972152 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 16:20:53.972225 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 16:20:53.972303 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":101,"threads_started":1}
I20260806 16:20:53.973583 3528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c1767aeb1e49ab92a8d482e738d2b7" candidate_uuid: "51943d5d2bb04e778a7b24de28590394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "175ae1a9d17341c09827d6d11d712e75" is_pre_election: true
I20260806 16:20:53.973560 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c1767aeb1e49ab92a8d482e738d2b7" candidate_uuid: "51943d5d2bb04e778a7b24de28590394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "732296a961b14967a5e2a6505fba1ba5" is_pre_election: true
I20260806 16:20:53.974012 3150 raft_consensus.cc:2454] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51943d5d2bb04e778a7b24de28590394 in term 0.
I20260806 16:20:53.974014 3528 raft_consensus.cc:2454] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51943d5d2bb04e778a7b24de28590394 in term 0.
I20260806 16:20:53.974679 3250 leader_election.cc:302] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 175ae1a9d17341c09827d6d11d712e75, 51943d5d2bb04e778a7b24de28590394; no voters:
I20260806 16:20:53.975085 3621 raft_consensus.cc:2781] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:20:53.975195 3621 raft_consensus.cc:482] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:20:53.975261 3621 raft_consensus.cc:3037] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:53.977147 3621 raft_consensus.cc:504] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } }
I20260806 16:20:53.977643 3621 leader_election.cc:288] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [CANDIDATE]: Term 1 election: Requested vote from peers 175ae1a9d17341c09827d6d11d712e75 (127.11.245.131:40507), 732296a961b14967a5e2a6505fba1ba5 (127.11.245.129:34733)
I20260806 16:20:53.978363 3528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c1767aeb1e49ab92a8d482e738d2b7" candidate_uuid: "51943d5d2bb04e778a7b24de28590394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "175ae1a9d17341c09827d6d11d712e75"
I20260806 16:20:53.978397 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c1767aeb1e49ab92a8d482e738d2b7" candidate_uuid: "51943d5d2bb04e778a7b24de28590394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "732296a961b14967a5e2a6505fba1ba5"
I20260806 16:20:53.978552 3528 raft_consensus.cc:3037] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:53.978570 3150 raft_consensus.cc:3037] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:20:53.979883 3150 raft_consensus.cc:2454] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51943d5d2bb04e778a7b24de28590394 in term 1.
I20260806 16:20:53.979887 3528 raft_consensus.cc:2454] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51943d5d2bb04e778a7b24de28590394 in term 1.
I20260806 16:20:53.980281 3250 leader_election.cc:302] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51943d5d2bb04e778a7b24de28590394, 732296a961b14967a5e2a6505fba1ba5; no voters:
I20260806 16:20:53.980531 3621 raft_consensus.cc:2781] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:20:53.980804 3621 raft_consensus.cc:686] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [term 1 LEADER]: Becoming Leader. State: Replica: 51943d5d2bb04e778a7b24de28590394, State: Running, Role: LEADER
I20260806 16:20:53.981259 3621 consensus_queue.cc:237] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } }
I20260806 16:20:53.985908 2943 catalog_manager.cc:5495] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51943d5d2bb04e778a7b24de28590394 (127.11.245.130). New cstate: current_term: 1 leader_uuid: "51943d5d2bb04e778a7b24de28590394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } health_report { overall_health: UNKNOWN } } }
I20260806 16:20:54.085059 3609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:54.081095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48962 (local address 127.11.245.130:44749)
0806 16:20:54.081288 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 16:20:54.081292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:20:54.081307 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:20:54.081419 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:20:54.081428 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:20:54.081504 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:20:54.081613 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:20:54.081862 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:54.082449 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:54.083266 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:20:54.083501 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:20:54.083584 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:20:54.083902 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:20:54.084588 (+ 686us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:20:54.084695 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 16:20:54.084779 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 16:20:54.084838 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 16:20:54.093667 3150 raft_consensus.cc:1261] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Refusing update from remote peer 51943d5d2bb04e778a7b24de28590394: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:20:54.093670 3528 raft_consensus.cc:1261] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75 [term 1 FOLLOWER]: Refusing update from remote peer 51943d5d2bb04e778a7b24de28590394: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:20:54.094650 3625 consensus_queue.cc:1035] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:20:54.094952 3621 consensus_queue.cc:1035] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:20:54.103173 3631 mvcc.cc:205] Tried to move back new op lower bound from 7315592208756510720 to 7315592208310870016. Current Snapshot: MvccSnapshot[applied={T|T < 7315592208756510720}]
I20260806 16:20:54.106081 3630 mvcc.cc:205] Tried to move back new op lower bound from 7315592208756510720 to 7315592208310870016. Current Snapshot: MvccSnapshot[applied={T|T < 7315592208756510720}]
I20260806 16:20:54.116331 2942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:20:54.116649 2942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:20:54.121269 2942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Sending DeleteTablet for 3 replicas of tablet d4c1767aeb1e49ab92a8d482e738d2b7
I20260806 16:20:54.123281 3124 tablet_service.cc:1514] Processing DeleteTablet for tablet d4c1767aeb1e49ab92a8d482e738d2b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:20:54 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:50804
I20260806 16:20:54.123384 3318 tablet_service.cc:1514] Processing DeleteTablet for tablet d4c1767aeb1e49ab92a8d482e738d2b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:20:54 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:48948
I20260806 16:20:54.123613 3503 tablet_service.cc:1514] Processing DeleteTablet for tablet d4c1767aeb1e49ab92a8d482e738d2b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:20:54 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:40532
I20260806 16:20:54.124172 3645 tablet_replica.cc:330] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394: stopping tablet replica
I20260806 16:20:54.124249 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 3004
I20260806 16:20:54.142064 3648 tablet_replica.cc:330] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75: stopping tablet replica
I20260806 16:20:54.142818 3648 raft_consensus.cc:2229] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:20:54.143519 3648 raft_consensus.cc:2258] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:20:54.146803 3648 ts_tablet_manager.cc:1905] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:20:54.151404 3648 ts_tablet_manager.cc:1918] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:20:54.151575 3648 log.cc:1196] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/d4c1767aeb1e49ab92a8d482e738d2b7
I20260806 16:20:54.152122 3648 ts_tablet_manager.cc:1939] T d4c1767aeb1e49ab92a8d482e738d2b7 P 175ae1a9d17341c09827d6d11d712e75: Deleting consensus metadata
I20260806 16:20:54.156695 2929 catalog_manager.cc:4841] TS 175ae1a9d17341c09827d6d11d712e75 (127.11.245.131:40507): tablet d4c1767aeb1e49ab92a8d482e738d2b7 (table test-table [id=85d74d7af2f84ff48cc8d7c7aadd32cc]) successfully deleted
I20260806 16:20:54.156967 3645 raft_consensus.cc:2229] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:20:54.157651 3645 raft_consensus.cc:2258] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:20:54.159826 3645 ts_tablet_manager.cc:1905] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:20:54.163960 3645 ts_tablet_manager.cc:1918] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:20:54.164240 3645 log.cc:1196] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/d4c1767aeb1e49ab92a8d482e738d2b7
I20260806 16:20:54.164685 3645 ts_tablet_manager.cc:1939] T d4c1767aeb1e49ab92a8d482e738d2b7 P 51943d5d2bb04e778a7b24de28590394: Deleting consensus metadata
I20260806 16:20:54.166818 2928 catalog_manager.cc:4841] TS 51943d5d2bb04e778a7b24de28590394 (127.11.245.130:44749): tablet d4c1767aeb1e49ab92a8d482e738d2b7 (table test-table [id=85d74d7af2f84ff48cc8d7c7aadd32cc]) successfully deleted
W20260806 16:20:54.167282 2929 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:34733 (error 108)
I20260806 16:20:54.167650 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 3220
I20260806 16:20:54.176748 3651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:20:54.175960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:34733 (local address 127.0.0.1:50818)
0806 16:20:54.176405 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 16:20:54.176546 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:34733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":170,"threads_started":1}
W20260806 16:20:54.177079 2929 catalog_manager.cc:4584] TS 732296a961b14967a5e2a6505fba1ba5 (127.11.245.129:34733): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d4c1767aeb1e49ab92a8d482e738d2b7: Network error: Client connection negotiation failed: client connection to 127.11.245.129:34733: connect: Connection refused (error 111)
I20260806 16:20:54.185446 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 3413
I20260806 16:20:54.208045 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 2901
I20260806 16:21:14.234602 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:46857
--webserver_interface=127.11.245.190
--webserver_port=36501
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:46857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:14.405735 3652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:14.406154 3652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:14.406206 3652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:14.411088 3652 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 16:21:14.411198 3652 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 16:21:14.411217 3652 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 16:21:14.411232 3652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:21:14.411254 3652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:14.411274 3652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:14.411363 3652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:21:14.411389 3652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:21:14.418027 3652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:46857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:46857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=36501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:14.421917 3652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:14.423512 3652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:21:14.436285 3652 server_base.cc:1034] running on GCE node
W20260806 16:21:14.436230 3663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:14.436180 3659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:14.436180 3660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:14.437598 3652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:14.439502 3652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:14.443099 3652 hybrid_clock.cc:648] HybridClock initialized: now 1786033274442986 us; error 93 us; skew 500 ppm
Aug 06 16:21:14 dist-test-slave-t3q3 krb5kdc[2109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033274, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:14.454751 3652 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:21:14.456535 3652 webserver.cc:460] Webserver started at http://127.11.245.190:36501/ using document root <none> and password file <none>
I20260806 16:21:14.457464 3652 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:14.457597 3652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:14.463500 3652 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:21:14.466847 3675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:14.468878 3652 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 16:21:14.469094 3652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "36797ff4d85440b78deada47e23ef7ce"
format_stamp: "Formatted at 2026-08-06 16:20:30 on dist-test-slave-t3q3"
server_key: "d8c2ff7a3f2f698df5ecf8b6ef9399de"
server_key_iv: "cbb0558cd6f955ae4eaf004917ead08a"
server_key_version: "encryptionkey@0"
I20260806 16:21:14.469962 3652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:14.489569 3652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:14.494611 3652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:14.494951 3652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:14.505298 3652 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:46857
I20260806 16:21:14.505801 3746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:46857 every 8 connection(s)
I20260806 16:21:14.507102 3652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:21:14.510986 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 3652
I20260806 16:21:14.511672 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:34733
--local_ip_for_outbound_sockets=127.11.245.129
--tserver_master_addrs=127.11.245.190:46857
--webserver_port=41653
--webserver_interface=127.11.245.129
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:21:14.513975 3747 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:21:14.517305 3747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Bootstrap starting.
I20260806 16:21:14.543185 3747 log.cc:824] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:14.561094 3747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:21:14.561664 3747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Bootstrap complete.
I20260806 16:21:14.566845 3747 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } }
I20260806 16:21:14.567384 3747 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36797ff4d85440b78deada47e23ef7ce, State: Initialized, Role: FOLLOWER
I20260806 16:21:14.568250 3747 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } }
I20260806 16:21:14.568462 3747 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:21:14.568558 3747 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:21:14.568713 3747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 2 FOLLOWER]: Advancing to term 3
I20260806 16:21:14.571383 3747 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } }
I20260806 16:21:14.571851 3747 leader_election.cc:302] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36797ff4d85440b78deada47e23ef7ce; no voters:
I20260806 16:21:14.572386 3747 leader_election.cc:288] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 16:21:14.572511 3752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 3 FOLLOWER]: Leader election won for term 3
I20260806 16:21:14.572722 3752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [term 3 LEADER]: Becoming Leader. State: Replica: 36797ff4d85440b78deada47e23ef7ce, State: Running, Role: LEADER
I20260806 16:21:14.573109 3752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } }
I20260806 16:21:14.574065 3747 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:21:14.574968 3753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36797ff4d85440b78deada47e23ef7ce. Latest consensus state: current_term: 3 leader_uuid: "36797ff4d85440b78deada47e23ef7ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } } }
I20260806 16:21:14.574972 3754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "36797ff4d85440b78deada47e23ef7ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36797ff4d85440b78deada47e23ef7ce" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46857 } } }
I20260806 16:21:14.575387 3754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:14.575749 3753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:14.576685 3765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:21:14.584705 3765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:21:14.585566 3765 catalog_manager.cc:1259] Loaded cluster ID: 3bacf4970a3c45c88a923244fd5637e3
I20260806 16:21:14.585641 3765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:21:14.588652 3765 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:21:14.589694 3765 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Loaded TSK: 0
I20260806 16:21:14.589798 3765 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 36797ff4d85440b78deada47e23ef7ce: Loaded TSK: 1
F20260806 16:21:14.590730 3765 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9e6b0899ad google::LogMessage::Fail() at ??:0
@ 0x7f9e6b08c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f9e6b0894de google::LogMessage::Flush() at ??:0
@ 0x7f9e6b08aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9e6dd4d18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f9e6dd45f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f9e759f27bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f9e759cc99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f9e759cd178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f9e75a0edf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f9e75a2d82b std::function<>::operator()() at ??:0
@ 0x7f9e759ccb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f9e759cdf3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9e759ca579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f9e75a0e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9e6d2db3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9e6d2dbc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f9e6d2dd3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9e6d2cc512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9e6e9106db start_thread at ??:0
@ 0x7f9e69b8761f clone at ??:0
W20260806 16:21:14.728271 3749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:14.728991 3749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:14.729128 3749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:14.736759 3749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:14.736918 3749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:14.737046 3749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:21:14.746990 3749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:34733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=41653
--tserver_master_addrs=127.11.245.190:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:14.749130 3749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:14.751016 3749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:14.766065 3812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:14.766479 3749 server_base.cc:1034] running on GCE node
W20260806 16:21:14.766029 3814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:14.766031 3811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:14.767184 3749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:14.768193 3749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:14.771593 3749 hybrid_clock.cc:648] HybridClock initialized: now 1786033274771494 us; error 116 us; skew 500 ppm
Aug 06 16:21:14 dist-test-slave-t3q3 krb5kdc[2109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033274, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:14.775674 3749 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:21:14.777024 3749 webserver.cc:460] Webserver started at http://127.11.245.129:41653/ using document root <none> and password file <none>
I20260806 16:21:14.777856 3749 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:14.777947 3749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:14.783576 3749 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:21:14.786731 3825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:14.789237 3749 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:21:14.789620 3749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "732296a961b14967a5e2a6505fba1ba5"
format_stamp: "Formatted at 2026-08-06 16:20:31 on dist-test-slave-t3q3"
server_key: "d3f588f0a1b8d17aba4282ba855a87ab"
server_key_iv: "cbd0af4711cd3e570674beabbf879ce9"
server_key_version: "encryptionkey@0"
I20260806 16:21:14.790455 3749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:14.811728 3749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:14.817502 3749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:14.817817 3749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:14.818725 3749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:14.820426 3833 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 16:21:14.828644 3749 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 16:21:14.828799 3749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.001s sys 0.000s
I20260806 16:21:14.828930 3749 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 16:21:14.831403 3749 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 16:21:14.831511 3749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260806 16:21:14.831974 3833 tablet_bootstrap.cc:492] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5: Bootstrap starting.
I20260806 16:21:14.845947 3749 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:34733
I20260806 16:21:14.846289 3994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:34733 every 8 connection(s)
I20260806 16:21:14.847535 3749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:21:14.856997 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 3749
I20260806 16:21:14.857697 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:44749
--local_ip_for_outbound_sockets=127.11.245.130
--tserver_master_addrs=127.11.245.190:46857
--webserver_port=45981
--webserver_interface=127.11.245.130
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:21:14.862206 3833 log.cc:824] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:14.876243 3833 tablet_bootstrap.cc:492] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:21:14.876904 3833 tablet_bootstrap.cc:492] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5: Bootstrap complete.
I20260806 16:21:14.879016 3833 ts_tablet_manager.cc:1397] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5: Time spent bootstrapping tablet: real 0.047s user 0.022s sys 0.022s
I20260806 16:21:14.884851 3833 raft_consensus.cc:348] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } }
I20260806 16:21:14.885730 3833 raft_consensus.cc:729] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 732296a961b14967a5e2a6505fba1ba5, State: Initialized, Role: FOLLOWER
I20260806 16:21:14.886602 3833 consensus_queue.cc:260] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } }
I20260806 16:21:14.888181 3833 ts_tablet_manager.cc:1428] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5: Time spent starting tablet: real 0.009s user 0.012s sys 0.000s
W20260806 16:21:14.933427 3998 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:21:14.850339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:33191)
0806 16:21:14.851142 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:14.851221 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:14.852419 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:14.931980 (+ 79561us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":184,"threads_started":1}
W20260806 16:21:14.934285 3995 heartbeater.cc:644] Failed to heartbeat to 127.11.245.190:46857 (0 consecutive failures): Network error: Failed to ping master at 127.11.245.190:46857: Client connection negotiation failed: client connection to 127.11.245.190:46857: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 16:21:14.935387 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:14.935012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:54169)
0806 16:21:14.935194 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:14.935298 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 16:21:14.936460 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:14.936209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:53575)
0806 16:21:14.936342 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:14.936398 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 16:21:14.936757 3995 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:21:15.026520 3999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:15.026973 3999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:15.027064 3999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:15.032379 3999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:15.032577 3999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:15.032704 3999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:21:15.039654 3999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:44749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=45981
--tserver_master_addrs=127.11.245.190:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:15.041503 3999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:15.043066 3999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:15.056540 4014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:15.056600 4011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:15.056609 4012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:15.057215 3999 server_base.cc:1034] running on GCE node
I20260806 16:21:15.057914 3999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:15.058840 3999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:15.060163 3999 hybrid_clock.cc:648] HybridClock initialized: now 1786033275060147 us; error 36 us; skew 500 ppm
Aug 06 16:21:15 dist-test-slave-t3q3 krb5kdc[2109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033275, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:15.063935 3999 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:21:15.065302 3999 webserver.cc:460] Webserver started at http://127.11.245.130:45981/ using document root <none> and password file <none>
I20260806 16:21:15.066058 3999 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:15.066128 3999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:15.071281 3999 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 16:21:15.074307 4023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:15.076123 3999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:21:15.076289 3999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "51943d5d2bb04e778a7b24de28590394"
format_stamp: "Formatted at 2026-08-06 16:20:31 on dist-test-slave-t3q3"
server_key: "82da88dc13669f9347fb54acfec7b349"
server_key_iv: "1a32a126b8660a51ed8817a1a174c4dc"
server_key_version: "encryptionkey@0"
I20260806 16:21:15.077019 3999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:15.091416 3999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:15.095880 3999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:15.096163 3999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:15.097023 3999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:15.098189 3999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:15.098244 3999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:15.098294 3999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:15.098352 3999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:15.112270 3999 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:44749
I20260806 16:21:15.112315 4188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:44749 every 8 connection(s)
I20260806 16:21:15.113775 3999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:21:15.120234 4192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:15.117268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:36881)
0806 16:21:15.118290 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:15.118467 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":192,"threads_started":1}
W20260806 16:21:15.121045 4189 heartbeater.cc:644] Failed to heartbeat to 127.11.245.190:46857 (0 consecutive failures): Network error: Failed to ping master at 127.11.245.190:46857: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
I20260806 16:21:15.121980 4192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:15.121643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:60077)
0806 16:21:15.121814 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:15.121891 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 16:21:15.122560 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 3999
I20260806 16:21:15.122792 4192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:15.122546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:60889)
0806 16:21:15.122678 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:15.122735 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 16:21:15.122973 4189 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 16:21:15.123246 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:40507
--local_ip_for_outbound_sockets=127.11.245.131
--tserver_master_addrs=127.11.245.190:46857
--webserver_port=33937
--webserver_interface=127.11.245.131
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:15.275981 4193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:15.276484 4193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:15.276580 4193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:15.281529 4193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:15.281643 4193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:15.281746 4193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:21:15.288731 4193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:40507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=33937
--tserver_master_addrs=127.11.245.190:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:15.290511 4193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:15.292140 4193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:15.306779 4200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:15.306779 4204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:15.306875 4193 server_base.cc:1034] running on GCE node
W20260806 16:21:15.306779 4202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:15.307729 4193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:15.308718 4193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:15.309981 4193 hybrid_clock.cc:648] HybridClock initialized: now 1786033275309948 us; error 40 us; skew 500 ppm
Aug 06 16:21:15 dist-test-slave-t3q3 krb5kdc[2109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033275, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:15.314478 4193 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:21:15.316179 4193 webserver.cc:460] Webserver started at http://127.11.245.131:33937/ using document root <none> and password file <none>
I20260806 16:21:15.317095 4193 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:15.317229 4193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:15.325433 4193 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:21:15.329478 4219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:15.331640 4193 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:21:15.332018 4193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "175ae1a9d17341c09827d6d11d712e75"
format_stamp: "Formatted at 2026-08-06 16:20:32 on dist-test-slave-t3q3"
server_key: "2048e5e336187b2be8e82d3efd0ff589"
server_key_iv: "ee445ec19ef5213a2c151fc88676087a"
server_key_version: "encryptionkey@0"
I20260806 16:21:15.332967 4193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:15.357622 4193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:15.364424 4193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:15.364738 4193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:15.365904 4193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:15.367830 4193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:15.367980 4193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:15.368126 4193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:15.368191 4193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:15.387156 4193 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:40507
I20260806 16:21:15.387174 4375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:40507 every 8 connection(s)
I20260806 16:21:15.388991 4193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:21:15.395283 4382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:15.392383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:41945)
0806 16:21:15.393298 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:15.393463 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":362,"threads_started":1}
W20260806 16:21:15.395983 4377 heartbeater.cc:644] Failed to heartbeat to 127.11.245.190:46857 (0 consecutive failures): Network error: Failed to ping master at 127.11.245.190:46857: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
I20260806 16:21:15.396574 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 4193
I20260806 16:21:15.397109 4382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:15.396753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:45015)
0806 16:21:15.396924 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:15.397009 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 16:21:15.398289 4382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:15.397658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:37861)
0806 16:21:15.398130 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:15.398215 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260806 16:21:15.398625 4377 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 16:21:15.938601 4384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:15.937873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:41639)
0806 16:21:15.938353 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:15.938433 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":165,"threads_started":1}
I20260806 16:21:16.124527 4385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:16.123842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:47705)
0806 16:21:16.124284 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:16.124363 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":120,"threads_started":1}
I20260806 16:21:16.141256 4388 raft_consensus.cc:482] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:21:16.141637 4388 raft_consensus.cc:504] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } }
I20260806 16:21:16.143488 4388 leader_election.cc:288] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51943d5d2bb04e778a7b24de28590394 (127.11.245.130:44749), 175ae1a9d17341c09827d6d11d712e75 (127.11.245.131:40507)
Aug 06 16:21:16 dist-test-slave-t3q3 krb5kdc[2109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033274, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.131@KRBTEST.COM
Aug 06 16:21:16 dist-test-slave-t3q3 krb5kdc[2109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033274, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.130@KRBTEST.COM
I20260806 16:21:16.166564 4389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:16.144143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:44749 (local address 127.11.245.129:57155)
0806 16:21:16.144681 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:16.144696 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:16.144794 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:16.147053 (+ 2259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:16.147062 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:16.147885 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:16.148598 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:16.148624 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:16.150620 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:16.150623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:16.151812 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:16.151828 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:16.152974 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:16.152979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:16.153768 (+ 789us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:16.154886 (+ 1118us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:21:16.154910 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:16.158488 (+ 3578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:16.162304 (+ 3816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:16.162319 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:16.162347 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:16.163525 (+ 1178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:16.164189 (+ 664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:16.164193 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:16.164195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:16.164347 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:16.165401 (+ 1054us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:16.165405 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:16.165715 (+ 310us) client_negotiation.cc:769] Sending connection context
0806 16:21:16.165962 (+ 247us) client_negotiation.cc:240] Negotiation successful
0806 16:21:16.166376 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":166,"spinlock_wait_cycles":6784,"thread_start_us":108,"threads_started":1}
I20260806 16:21:16.166740 4390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:16.144113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:44005 (local address 127.11.245.131:40507)
0806 16:21:16.144739 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:16.144765 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:16.144834 (+ 69us) server_negotiation.cc:407] Connection header received
0806 16:21:16.146097 (+ 1263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:16.146105 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:16.146511 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:16.146909 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:16.149024 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:16.150240 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:16.151953 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:16.152773 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:16.153056 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:16.158943 (+ 5887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:16.158953 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:16.158983 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:16.159042 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:16.162215 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:16.163578 (+ 1363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:16.163593 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:16.163605 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:16.163914 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:16.164482 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:16.164486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:16.164493 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:16.165167 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:16.165332 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:16.166146 (+ 814us) server_negotiation.cc:299] Negotiation successful
0806 16:21:16.166531 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":130,"threads_started":1}
I20260806 16:21:16.167068 4384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:16.143831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:40507 (local address 127.11.245.129:44005)
0806 16:21:16.144112 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:16.144134 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:16.144291 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:16.147074 (+ 2783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:16.147079 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:16.147881 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:16.148588 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:16.148612 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:16.150489 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:16.150495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:16.151735 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:16.151753 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:16.152987 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:16.152992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:16.153285 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:16.154222 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:21:16.154255 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:16.158088 (+ 3833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:16.162898 (+ 4810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:16.162902 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:16.162904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:16.163287 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:16.164054 (+ 767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:16.164088 (+ 34us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:16.164093 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:16.164317 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:16.165313 (+ 996us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:16.165321 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:16.165717 (+ 396us) client_negotiation.cc:769] Sending connection context
0806 16:21:16.165965 (+ 248us) client_negotiation.cc:240] Negotiation successful
0806 16:21:16.166495 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":325,"spinlock_wait_cycles":40960}
I20260806 16:21:16.167651 4391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:16.144371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:57155 (local address 127.11.245.130:44749)
0806 16:21:16.144866 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:16.144899 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:16.144956 (+ 57us) server_negotiation.cc:407] Connection header received
0806 16:21:16.145878 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:16.145886 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:16.146427 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:16.146909 (+ 482us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:16.148994 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:16.150341 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:16.152151 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:16.152770 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:16.153054 (+ 284us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:16.159657 (+ 6603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:16.159673 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:16.159695 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:16.159755 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:16.161933 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:16.163781 (+ 1848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:16.163791 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:16.163798 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:16.164037 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:16.164489 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:16.164493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:16.164495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:16.165152 (+ 657us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:16.165333 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:16.167157 (+ 1824us) server_negotiation.cc:299] Negotiation successful
0806 16:21:16.167486 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":216,"threads_started":1}
W20260806 16:21:16.170670 3827 leader_election.cc:341] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 51943d5d2bb04e778a7b24de28590394 (127.11.245.130:44749): Not found: Tablet not found: d4c1767aeb1e49ab92a8d482e738d2b7
W20260806 16:21:16.170889 3828 leader_election.cc:341] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 175ae1a9d17341c09827d6d11d712e75 (127.11.245.131:40507): Not found: Tablet not found: d4c1767aeb1e49ab92a8d482e738d2b7
I20260806 16:21:16.170974 3827 leader_election.cc:302] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 732296a961b14967a5e2a6505fba1ba5; no voters: 175ae1a9d17341c09827d6d11d712e75, 51943d5d2bb04e778a7b24de28590394
I20260806 16:21:16.171388 4388 raft_consensus.cc:2726] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:21:16.400027 4394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:16.399423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:51439)
0806 16:21:16.399801 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:16.399887 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":95,"threads_started":1}
I20260806 16:21:16.940675 4395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:16.939956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:46671)
0806 16:21:16.940415 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:16.940524 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":190,"threads_started":1}
I20260806 16:21:17.126507 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:17.125823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:34977)
0806 16:21:17.126247 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:17.126338 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":120,"threads_started":1}
I20260806 16:21:17.402599 4397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:17.401748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:47613)
0806 16:21:17.402291 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:17.402393 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":237,"threads_started":1}
I20260806 16:21:17.892771 4398 raft_consensus.cc:482] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:21:17.893038 4398 raft_consensus.cc:504] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } }
I20260806 16:21:17.893703 4398 leader_election.cc:288] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51943d5d2bb04e778a7b24de28590394 (127.11.245.130:44749), 175ae1a9d17341c09827d6d11d712e75 (127.11.245.131:40507)
W20260806 16:21:17.895025 3827 leader_election.cc:341] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 51943d5d2bb04e778a7b24de28590394 (127.11.245.130:44749): Not found: Tablet not found: d4c1767aeb1e49ab92a8d482e738d2b7
W20260806 16:21:17.895195 3828 leader_election.cc:341] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 175ae1a9d17341c09827d6d11d712e75 (127.11.245.131:40507): Not found: Tablet not found: d4c1767aeb1e49ab92a8d482e738d2b7
I20260806 16:21:17.895305 3828 leader_election.cc:302] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 732296a961b14967a5e2a6505fba1ba5; no voters: 175ae1a9d17341c09827d6d11d712e75, 51943d5d2bb04e778a7b24de28590394
I20260806 16:21:17.895503 4398 raft_consensus.cc:2726] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:21:17.943027 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:17.942219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:52759)
0806 16:21:17.942789 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:17.942867 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":356,"threads_started":1}
I20260806 16:21:18.128599 4402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:18.127892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:44455)
0806 16:21:18.128339 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:18.128448 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":175,"threads_started":1}
I20260806 16:21:18.404838 4403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:18.404048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:33391)
0806 16:21:18.404611 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:18.404695 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":158,"threads_started":1}
I20260806 16:21:18.945173 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:18.944515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:38829)
0806 16:21:18.944900 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:18.945005 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":122,"threads_started":1}
I20260806 16:21:19.130465 4407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:19.129828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:54113)
0806 16:21:19.130233 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:19.130321 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":170,"threads_started":1}
I20260806 16:21:19.407003 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:19.406334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:57441)
0806 16:21:19.406783 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:19.406863 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":150,"threads_started":1}
I20260806 16:21:19.947086 4409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:19.946469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:34997)
0806 16:21:19.946884 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:19.946958 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":106,"threads_started":1}
I20260806 16:21:20.132519 4410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:20.131759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:57399)
0806 16:21:20.132276 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:20.132383 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":163,"threads_started":1}
I20260806 16:21:20.288460 4414 raft_consensus.cc:482] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:21:20.288880 4414 raft_consensus.cc:504] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } }
I20260806 16:21:20.289567 4414 leader_election.cc:288] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51943d5d2bb04e778a7b24de28590394 (127.11.245.130:44749), 175ae1a9d17341c09827d6d11d712e75 (127.11.245.131:40507)
W20260806 16:21:20.290758 3827 leader_election.cc:341] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 51943d5d2bb04e778a7b24de28590394 (127.11.245.130:44749): Not found: Tablet not found: d4c1767aeb1e49ab92a8d482e738d2b7
W20260806 16:21:20.291029 3828 leader_election.cc:341] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 175ae1a9d17341c09827d6d11d712e75 (127.11.245.131:40507): Not found: Tablet not found: d4c1767aeb1e49ab92a8d482e738d2b7
I20260806 16:21:20.291152 3828 leader_election.cc:302] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 732296a961b14967a5e2a6505fba1ba5; no voters: 175ae1a9d17341c09827d6d11d712e75, 51943d5d2bb04e778a7b24de28590394
I20260806 16:21:20.291339 4414 raft_consensus.cc:2726] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:21:20.409060 4415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:20.408425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:53677)
0806 16:21:20.408813 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:20.408904 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":95,"threads_started":1}
I20260806 16:21:20.948987 4417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:20.948368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:48753)
0806 16:21:20.948766 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:20.948844 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":111,"threads_started":1}
I20260806 16:21:21.134409 4418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:21.133772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:50215)
0806 16:21:21.134166 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:21.134247 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":122,"threads_started":1}
I20260806 16:21:21.410888 4419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:21.410289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:33295)
0806 16:21:21.410664 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:21.410742 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":107,"threads_started":1}
I20260806 16:21:21.951085 4420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:21.950424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:37863)
0806 16:21:21.950847 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:21.950943 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":141,"threads_started":1}
I20260806 16:21:22.136358 4423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:22.135471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:59385)
0806 16:21:22.135908 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:22.135982 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":131,"threads_started":1}
I20260806 16:21:22.413041 4425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:22.412328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:59465)
0806 16:21:22.412784 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:22.412863 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":107,"threads_started":1}
I20260806 16:21:22.953125 4426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:22.952478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:46789)
0806 16:21:22.952882 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:22.952971 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":113,"threads_started":1}
I20260806 16:21:23.138742 4427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:23.138026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:32897)
0806 16:21:23.138522 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:23.138611 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":262,"threads_started":1}
I20260806 16:21:23.414791 4428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:23.414233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:47681)
0806 16:21:23.414584 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:23.414659 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":108,"threads_started":1}
I20260806 16:21:23.955313 4430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:23.954260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:39777)
0806 16:21:23.954654 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:23.954731 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":136,"threads_started":1}
I20260806 16:21:24.140834 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:24.140033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:55085)
0806 16:21:24.140593 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:24.140698 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":139,"threads_started":1}
I20260806 16:21:24.182904 4432 raft_consensus.cc:482] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:21:24.183156 4432 raft_consensus.cc:504] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51943d5d2bb04e778a7b24de28590394" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 44749 } } peers { permanent_uuid: "175ae1a9d17341c09827d6d11d712e75" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40507 } } peers { permanent_uuid: "732296a961b14967a5e2a6505fba1ba5" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 34733 } }
I20260806 16:21:24.184509 4432 leader_election.cc:288] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51943d5d2bb04e778a7b24de28590394 (127.11.245.130:44749), 175ae1a9d17341c09827d6d11d712e75 (127.11.245.131:40507)
W20260806 16:21:24.185509 3827 leader_election.cc:341] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 51943d5d2bb04e778a7b24de28590394 (127.11.245.130:44749): Not found: Tablet not found: d4c1767aeb1e49ab92a8d482e738d2b7
W20260806 16:21:24.185647 3828 leader_election.cc:341] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 175ae1a9d17341c09827d6d11d712e75 (127.11.245.131:40507): Not found: Tablet not found: d4c1767aeb1e49ab92a8d482e738d2b7
I20260806 16:21:24.185747 3828 leader_election.cc:302] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 732296a961b14967a5e2a6505fba1ba5; no voters: 175ae1a9d17341c09827d6d11d712e75, 51943d5d2bb04e778a7b24de28590394
I20260806 16:21:24.185906 4432 raft_consensus.cc:2726] T d4c1767aeb1e49ab92a8d482e738d2b7 P 732296a961b14967a5e2a6505fba1ba5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:21:24.416788 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:24.416181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:59039)
0806 16:21:24.416573 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:24.416652 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":134,"threads_started":1}
I20260806 16:21:24.957708 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:24.956945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:54915)
0806 16:21:24.957436 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:24.957544 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":130,"threads_started":1}
I20260806 16:21:25.142997 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:25.142259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:34485)
0806 16:21:25.142776 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:25.142861 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":117,"threads_started":1}
I20260806 16:21:25.418480 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:25.417866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:46181)
0806 16:21:25.418256 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:25.418350 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":129,"threads_started":1}
I20260806 16:21:25.959408 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:25.958794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:45121)
0806 16:21:25.959180 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:25.959252 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":145,"threads_started":1}
I20260806 16:21:26.144786 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:26.144241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:55471)
0806 16:21:26.144600 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:26.144672 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":97,"threads_started":1}
I20260806 16:21:26.420282 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:26.419564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:33771)
0806 16:21:26.419967 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:26.420039 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":134,"threads_started":1}
I20260806 16:21:26.961066 4442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:26.960487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:37985)
0806 16:21:26.960865 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:26.960938 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":137,"threads_started":1}
I20260806 16:21:27.146852 4443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:27.146032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:40365)
0806 16:21:27.146512 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:27.146611 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":116,"threads_started":1}
I20260806 16:21:27.421970 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:27.421401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:52185)
0806 16:21:27.421749 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:27.421833 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":114,"threads_started":1}
I20260806 16:21:27.962785 4445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:27.962233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:60457)
0806 16:21:27.962585 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:27.962654 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":122,"threads_started":1}
I20260806 16:21:28.148813 4446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:28.148234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:38299)
0806 16:21:28.148607 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:28.148678 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":109,"threads_started":1}
I20260806 16:21:28.423779 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:28.423046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:53859)
0806 16:21:28.423584 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:28.423651 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":143,"threads_started":1}
I20260806 16:21:28.964638 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:28.963960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:58557)
0806 16:21:28.964364 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:28.964449 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":170,"threads_started":1}
I20260806 16:21:29.150470 4450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:29.149885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:55775)
0806 16:21:29.150254 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:29.150328 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":111,"threads_started":1}
I20260806 16:21:29.425768 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:29.425155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:53641)
0806 16:21:29.425557 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:29.425633 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":127,"threads_started":1}
I20260806 16:21:29.966637 4453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:29.966093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.129:51713)
0806 16:21:29.966452 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:29.966515 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":114,"threads_started":1}
I20260806 16:21:30.152449 4454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:30.151812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.130:58205)
0806 16:21:30.152256 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:30.152321 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":116,"threads_started":1}
I20260806 16:21:30.406716 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 3749
I20260806 16:21:30.427707 4455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:30.427162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:46857 (local address 127.11.245.131:59999)
0806 16:21:30.427506 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:30.427577 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.190:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":127,"threads_started":1}
I20260806 16:21:30.430027 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 3999
I20260806 16:21:30.447160 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 4193
2026-08-06T16:21:30Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (61218 ms)
[----------] 30 tests from SecurityITest (279270 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:21:30 dist-test-slave-t3q3 krb5kdc[4462](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:21:30 dist-test-slave-t3q3 krb5kdc[4462](info): set up 2 sockets
Aug 06 16:21:30 dist-test-slave-t3q3 krb5kdc[4462](info): commencing operation
krb5kdc: starting...
W20260806 16:21:31.775046 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:21:31 dist-test-slave-t3q3 krb5kdc[4462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:21:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:21:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:21:31.944720 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:35599
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:34931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:35599
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:32.098186 4481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:32.098694 4481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:32.098801 4481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:32.103389 4481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:21:32.103511 4481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:32.103552 4481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:32.103606 4481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:21:32.103663 4481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:21:32.110256 4481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:34931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:35599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:35599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:32.112118 4481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:32.113605 4481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:32.125106 4487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:32.125243 4481 server_base.cc:1034] running on GCE node
W20260806 16:21:32.125334 4488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:32.125106 4491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:32.126583 4481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:32.128130 4481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:32.129383 4481 hybrid_clock.cc:648] HybridClock initialized: now 1786033292129362 us; error 41 us; skew 500 ppm
Aug 06 16:21:32 dist-test-slave-t3q3 krb5kdc[4462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033292, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:32.133329 4481 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:21:32.134634 4481 webserver.cc:460] Webserver started at http://127.11.245.190:40495/ using document root <none> and password file <none>
I20260806 16:21:32.135594 4481 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:32.135723 4481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:32.136053 4481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:32.138994 4481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "0b82cf6fd3774c98a19bba28d0ce3247"
format_stamp: "Formatted at 2026-08-06 16:21:32 on dist-test-slave-t3q3"
server_key: "8bba8ccd8e3ff82e9bab91780b0a1c47"
server_key_iv: "a396e7f408e4c749b05a87024e079f83"
server_key_version: "encryptionkey@0"
I20260806 16:21:32.139784 4481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "0b82cf6fd3774c98a19bba28d0ce3247"
format_stamp: "Formatted at 2026-08-06 16:21:32 on dist-test-slave-t3q3"
server_key: "8bba8ccd8e3ff82e9bab91780b0a1c47"
server_key_iv: "a396e7f408e4c749b05a87024e079f83"
server_key_version: "encryptionkey@0"
I20260806 16:21:32.145342 4481 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:21:32.149107 4501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:32.151356 4481 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:21:32.151599 4481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "0b82cf6fd3774c98a19bba28d0ce3247"
format_stamp: "Formatted at 2026-08-06 16:21:32 on dist-test-slave-t3q3"
server_key: "8bba8ccd8e3ff82e9bab91780b0a1c47"
server_key_iv: "a396e7f408e4c749b05a87024e079f83"
server_key_version: "encryptionkey@0"
I20260806 16:21:32.151798 4481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:32.173086 4481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:32.177924 4481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:32.178315 4481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:32.188025 4481 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:35599
I20260806 16:21:32.188177 4572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:35599 every 8 connection(s)
I20260806 16:21:32.189482 4481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:21:32.193701 4573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:32.197170 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 4481
I20260806 16:21:32.197360 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:21:32.197760 12246 external_mini_cluster.cc:1428] Setting key a190a6e7a415d204b181bb522120366d
I20260806 16:21:32.202296 4573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247: Bootstrap starting.
I20260806 16:21:32.205734 4573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:32.206768 4573 log.cc:824] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:21:32 dist-test-slave-t3q3 krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:21:32.209884 4573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247: No bootstrap required, opened a new log
I20260806 16:21:32.214504 4573 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b82cf6fd3774c98a19bba28d0ce3247" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 35599 } }
I20260806 16:21:32.214860 4573 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:32.214962 4573 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b82cf6fd3774c98a19bba28d0ce3247, State: Initialized, Role: FOLLOWER
I20260806 16:21:32.215878 4573 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b82cf6fd3774c98a19bba28d0ce3247" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 35599 } }
I20260806 16:21:32.216141 4573 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:21:32.216243 4573 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:21:32.216325 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:32.199968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52488 (local address 127.11.245.190:35599)
0806 16:21:32.200631 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:32.200651 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:32.200697 (+ 46us) server_negotiation.cc:407] Connection header received
0806 16:21:32.201486 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:32.201509 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:32.201943 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:32.202385 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:32.203465 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:32.204706 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:32.205708 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:32.206324 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:32.206538 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:32.210060 (+ 3522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:32.210069 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:32.210102 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:32.210139 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:32.212527 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:32.213104 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:32.213116 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:32.213123 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:32.213259 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:32.213585 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:32.213589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:32.213591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:32.214139 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:32.214382 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:32.214720 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 16:21:32.215139 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":184,"threads_started":1}
I20260806 16:21:32.216382 4573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:32.218009 4573 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b82cf6fd3774c98a19bba28d0ce3247" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 35599 } }
I20260806 16:21:32.218477 4573 leader_election.cc:302] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b82cf6fd3774c98a19bba28d0ce3247; no voters:
I20260806 16:21:32.218900 4573 leader_election.cc:288] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:21:32.219041 4582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:32.219288 4582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [term 1 LEADER]: Becoming Leader. State: Replica: 0b82cf6fd3774c98a19bba28d0ce3247, State: Running, Role: LEADER
I20260806 16:21:32.219761 4582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b82cf6fd3774c98a19bba28d0ce3247" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 35599 } }
I20260806 16:21:32.220605 4573 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:21:32.222021 4586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b82cf6fd3774c98a19bba28d0ce3247. Latest consensus state: current_term: 1 leader_uuid: "0b82cf6fd3774c98a19bba28d0ce3247" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b82cf6fd3774c98a19bba28d0ce3247" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 35599 } } }
I20260806 16:21:32.222208 4586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:32.222009 4584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b82cf6fd3774c98a19bba28d0ce3247" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b82cf6fd3774c98a19bba28d0ce3247" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 35599 } } }
I20260806 16:21:32.222393 4584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247 [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:32.222764 4598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:21:32.226881 4598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:21:32.234656 4598 catalog_manager.cc:1347] Generated new cluster ID: 5e0869047d664ed8a3d052a5c5be8def
I20260806 16:21:32.234778 4598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:21:32.249282 4598 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:21:32.251039 4598 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:21:32.261950 4598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247: Generated new TSK 0
I20260806 16:21:32.263173 4598 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:21:32.307610 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:35599
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:34931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:32.466049 4613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:32.466477 4613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:32.466534 4613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:32.471818 4613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:32.471930 4613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:32.472000 4613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:21:32.478549 4613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:34931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:35599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:32.480033 4613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:32.481454 4613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:32.494320 4626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:32.494316 4622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:32.494316 4628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:32.494948 4613 server_base.cc:1034] running on GCE node
I20260806 16:21:32.495533 4613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:32.496518 4613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:32.499894 4613 hybrid_clock.cc:648] HybridClock initialized: now 1786033292499826 us; error 57 us; skew 500 ppm
Aug 06 16:21:32 dist-test-slave-t3q3 krb5kdc[4462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033292, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:32.504199 4613 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:21:32.505594 4613 webserver.cc:460] Webserver started at http://127.11.245.129:45379/ using document root <none> and password file <none>
I20260806 16:21:32.506366 4613 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:32.506441 4613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:32.507057 4613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:32.510053 4613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "3ae86c82305d4a6ab1cd9f2b8bcc225f"
format_stamp: "Formatted at 2026-08-06 16:21:32 on dist-test-slave-t3q3"
server_key: "d851320504c0e37a948e59dfe897dd9a"
server_key_iv: "d1dcf969719ef36acc23e35e3c111955"
server_key_version: "encryptionkey@0"
I20260806 16:21:32.510778 4613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "3ae86c82305d4a6ab1cd9f2b8bcc225f"
format_stamp: "Formatted at 2026-08-06 16:21:32 on dist-test-slave-t3q3"
server_key: "d851320504c0e37a948e59dfe897dd9a"
server_key_iv: "d1dcf969719ef36acc23e35e3c111955"
server_key_version: "encryptionkey@0"
I20260806 16:21:32.516956 4613 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260806 16:21:32.521456 4638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:32.523566 4613 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:21:32.523783 4613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "3ae86c82305d4a6ab1cd9f2b8bcc225f"
format_stamp: "Formatted at 2026-08-06 16:21:32 on dist-test-slave-t3q3"
server_key: "d851320504c0e37a948e59dfe897dd9a"
server_key_iv: "d1dcf969719ef36acc23e35e3c111955"
server_key_version: "encryptionkey@0"
I20260806 16:21:32.523962 4613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:32.545876 4613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:32.551508 4613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:32.551813 4613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:32.552799 4613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:32.554006 4613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:32.554064 4613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:32.554112 4613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:32.554165 4613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:32.568343 4613 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:37499
I20260806 16:21:32.568370 4793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:37499 every 8 connection(s)
I20260806 16:21:32.569707 4613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:21:32.580093 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 4613
I20260806 16:21:32.580295 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:21:32.580732 12246 external_mini_cluster.cc:1428] Setting key f27b182f2eeac950bea473f5c2bdf7b0
Aug 06 16:21:32 dist-test-slave-t3q3 krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033292, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:21:32.590473 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:32.572723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:33335 (local address 127.11.245.190:35599)
0806 16:21:32.572912 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:32.572916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:32.574120 (+ 1204us) server_negotiation.cc:407] Connection header received
0806 16:21:32.576233 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:32.576246 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:32.576330 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:32.576426 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:32.578163 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:32.578817 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:32.580651 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:32.580981 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:32.581093 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:32.584983 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:32.584987 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:32.584990 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:32.585022 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:32.586705 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:32.587849 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:32.587945 (+ 96us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:32.587947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:32.588223 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:32.588753 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:32.588758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:32.588760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:32.589019 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:32.589157 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:32.589983 (+ 826us) server_negotiation.cc:299] Negotiation successful
0806 16:21:32.590219 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 16:21:32.591854 4796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:32.573174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:35599 (local address 127.11.245.129:33335)
0806 16:21:32.573888 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:32.573945 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:32.575754 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:32.576737 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:32.576747 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:32.577254 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:32.577947 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:32.577964 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:32.578985 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:32.578988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:32.580468 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:32.580486 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:32.581201 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:32.581206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:32.581445 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:32.582391 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:21:32.582416 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:32.584714 (+ 2298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:32.586876 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:32.586886 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:32.586917 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:32.587591 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:32.588388 (+ 797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:32.588392 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:32.588394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:32.588621 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:32.589181 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:32.589191 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:32.589596 (+ 405us) client_negotiation.cc:769] Sending connection context
0806 16:21:32.589862 (+ 266us) client_negotiation.cc:240] Negotiation successful
0806 16:21:32.590496 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":197,"threads_started":1}
I20260806 16:21:32.593933 4794 heartbeater.cc:347] Connected to a master server at 127.11.245.190:35599
I20260806 16:21:32.594334 4794 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:32.595393 4794 heartbeater.cc:511] Master 127.11.245.190:35599 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:21:32.598157 4523 ts_manager.cc:194] Registered new tserver with Master: 3ae86c82305d4a6ab1cd9f2b8bcc225f (127.11.245.129:37499)
I20260806 16:21:32.599694 4523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:33335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:21:32.631597 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:35599
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:34931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:32.799022 4799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:32.799407 4799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:32.799459 4799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:32.804028 4799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:32.804211 4799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:32.804291 4799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:21:32.810743 4799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:34931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:35599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:32.812448 4799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:32.813970 4799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:32.827111 4808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:32.827111 4812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:32.827098 4806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:32.827638 4799 server_base.cc:1034] running on GCE node
I20260806 16:21:32.828305 4799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:32.829578 4799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:32.831461 4799 hybrid_clock.cc:648] HybridClock initialized: now 1786033292831407 us; error 51 us; skew 500 ppm
Aug 06 16:21:32 dist-test-slave-t3q3 krb5kdc[4462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033292, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:32.836462 4799 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:21:32.838042 4799 webserver.cc:460] Webserver started at http://127.11.245.130:40087/ using document root <none> and password file <none>
I20260806 16:21:32.838927 4799 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:32.839206 4799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:32.839670 4799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:32.842712 4799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "35d85934c6134bea97f11f32619ed223"
format_stamp: "Formatted at 2026-08-06 16:21:32 on dist-test-slave-t3q3"
server_key: "a09f7654ca032bf7954c8e807ccc05b4"
server_key_iv: "b5f3be767a86da6e9a5d6234a1663903"
server_key_version: "encryptionkey@0"
I20260806 16:21:32.843514 4799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "35d85934c6134bea97f11f32619ed223"
format_stamp: "Formatted at 2026-08-06 16:21:32 on dist-test-slave-t3q3"
server_key: "a09f7654ca032bf7954c8e807ccc05b4"
server_key_iv: "b5f3be767a86da6e9a5d6234a1663903"
server_key_version: "encryptionkey@0"
I20260806 16:21:32.849541 4799 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 16:21:32.854228 4821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:32.856775 4799 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:21:32.857170 4799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "35d85934c6134bea97f11f32619ed223"
format_stamp: "Formatted at 2026-08-06 16:21:32 on dist-test-slave-t3q3"
server_key: "a09f7654ca032bf7954c8e807ccc05b4"
server_key_iv: "b5f3be767a86da6e9a5d6234a1663903"
server_key_version: "encryptionkey@0"
I20260806 16:21:32.857400 4799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:32.879737 4799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:32.884508 4799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:32.884806 4799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:32.885696 4799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:32.886878 4799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:32.886968 4799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:32.887058 4799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:32.887110 4799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:32.901903 4799 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:40925
I20260806 16:21:32.901963 4971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:40925 every 8 connection(s)
I20260806 16:21:32.903208 4799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:21:32.912447 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 4799
I20260806 16:21:32.912606 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:21:32.913089 12246 external_mini_cluster.cc:1428] Setting key 8ab55c7ee02901ddbf66a4aa56e62f9e
Aug 06 16:21:32 dist-test-slave-t3q3 krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033292, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:21:32.927062 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:32.906573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:49921 (local address 127.11.245.190:35599)
0806 16:21:32.906792 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:32.906796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:32.908418 (+ 1622us) server_negotiation.cc:407] Connection header received
0806 16:21:32.910430 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:32.910434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:32.910520 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:32.910624 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:32.912937 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:32.913735 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:32.915708 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:32.916287 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:32.916435 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:32.920607 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:32.920612 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:32.920615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:32.920653 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:32.922864 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:32.923627 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:32.923634 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:32.923639 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:32.923767 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:32.924846 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:32.924853 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:32.924859 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:32.925524 (+ 665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:32.925682 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:32.926575 (+ 893us) server_negotiation.cc:299] Negotiation successful
0806 16:21:32.926809 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 16:21:32.928329 4976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:32.907067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:35599 (local address 127.11.245.130:49921)
0806 16:21:32.907992 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:32.908163 (+ 171us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:32.910027 (+ 1864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:32.911136 (+ 1109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:32.911148 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:32.911679 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:32.912488 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:32.912511 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:32.913965 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:32.913970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:32.915441 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:32.915462 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:32.916572 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:32.916580 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:32.916935 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:32.917926 (+ 991us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:21:32.917961 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:32.920299 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:32.923061 (+ 2762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:32.923071 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:32.923097 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:32.923478 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:32.924004 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:32.924009 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:32.924011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:32.924684 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:32.925712 (+ 1028us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:32.925721 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:32.926162 (+ 441us) client_negotiation.cc:769] Sending connection context
0806 16:21:32.926452 (+ 290us) client_negotiation.cc:240] Negotiation successful
0806 16:21:32.926889 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":174,"threads_started":1}
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
I20260806 16:21:32.930564 4972 heartbeater.cc:347] Connected to a master server at 127.11.245.190:35599
I20260806 16:21:32.930941 4972 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:32.931752 4972 heartbeater.cc:511] Master 127.11.245.190:35599 requested a full tablet report, sending...
I20260806 16:21:32.933784 4523 ts_manager.cc:194] Registered new tserver with Master: 35d85934c6134bea97f11f32619ed223 (127.11.245.130:40925)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
I20260806 16:21:32.935218 4523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:49921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:21:32.961395 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:35599
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:34931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:33.115854 4979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:33.116321 4979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:33.116412 4979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:33.121098 4979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:33.121233 4979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:33.121340 4979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:21:33.129311 4979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:34931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:35599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:33.131059 4979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:33.132560 4979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:33.147130 4985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:33.147142 4986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:33.147142 4989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:33.147253 4979 server_base.cc:1034] running on GCE node
I20260806 16:21:33.148314 4979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:33.149287 4979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:33.152676 4979 hybrid_clock.cc:648] HybridClock initialized: now 1786033293152587 us; error 119 us; skew 500 ppm
Aug 06 16:21:33 dist-test-slave-t3q3 krb5kdc[4462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033293, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:33.157730 4979 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:21:33.159722 4979 webserver.cc:460] Webserver started at http://127.11.245.131:37639/ using document root <none> and password file <none>
I20260806 16:21:33.160657 4979 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:33.160746 4979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:33.161053 4979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:33.164673 4979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "6214b14b98cc42dda87a62e625f31d09"
format_stamp: "Formatted at 2026-08-06 16:21:33 on dist-test-slave-t3q3"
server_key: "0b8b082c7316ff38cd10ecd1e1254fcd"
server_key_iv: "37b157d6ea0f35b966fd30c36e4478a5"
server_key_version: "encryptionkey@0"
I20260806 16:21:33.165485 4979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "6214b14b98cc42dda87a62e625f31d09"
format_stamp: "Formatted at 2026-08-06 16:21:33 on dist-test-slave-t3q3"
server_key: "0b8b082c7316ff38cd10ecd1e1254fcd"
server_key_iv: "37b157d6ea0f35b966fd30c36e4478a5"
server_key_version: "encryptionkey@0"
I20260806 16:21:33.170811 4979 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:21:33.174605 5001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:33.178150 4979 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:21:33.178491 4979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "6214b14b98cc42dda87a62e625f31d09"
format_stamp: "Formatted at 2026-08-06 16:21:33 on dist-test-slave-t3q3"
server_key: "0b8b082c7316ff38cd10ecd1e1254fcd"
server_key_iv: "37b157d6ea0f35b966fd30c36e4478a5"
server_key_version: "encryptionkey@0"
I20260806 16:21:33.178747 4979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:33.198014 4979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:33.203845 4979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:33.204198 4979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:33.205137 4979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:33.206332 4979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:33.206388 4979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:33.206441 4979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:33.206457 4979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:33.220671 4979 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:35391
I20260806 16:21:33.220726 5158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:35391 every 8 connection(s)
I20260806 16:21:33.222169 4979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:21:33.223920 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 4979
I20260806 16:21:33.224129 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:21:33.224525 12246 external_mini_cluster.cc:1428] Setting key 21a12206593cd512e73ac6fbcb0f65e7
Aug 06 16:21:33 dist-test-slave-t3q3 krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033293, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:21:33.242913 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:33.225488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:35179 (local address 127.11.245.190:35599)
0806 16:21:33.225696 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:33.225702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:33.227218 (+ 1516us) server_negotiation.cc:407] Connection header received
0806 16:21:33.229356 (+ 2138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:33.229361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:33.229442 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:33.229539 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:33.231535 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.232243 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.233671 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.233931 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.234058 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:33.238079 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:33.238084 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:33.238087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:33.238124 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:33.240171 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:33.240811 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:33.240815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:33.240818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:33.240912 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:33.241448 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:33.241454 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:33.241456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:33.241741 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:33.241864 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:33.242557 (+ 693us) server_negotiation.cc:299] Negotiation successful
0806 16:21:33.242717 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 16:21:33.244287 5162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:33.226293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:35599 (local address 127.11.245.131:35179)
0806 16:21:33.226990 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:33.227077 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:33.228994 (+ 1917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:33.229825 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:33.229836 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:33.230436 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:33.231297 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:33.231324 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.232405 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.232409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:33.233520 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:33.233532 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.234137 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.234142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:33.234356 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:33.235157 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:21:33.235177 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:33.237774 (+ 2597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:33.240318 (+ 2544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:33.240327 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:33.240353 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:33.240692 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:33.241026 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:33.241030 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:33.241032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:33.241271 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:33.241868 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:33.241880 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:33.242257 (+ 377us) client_negotiation.cc:769] Sending connection context
0806 16:21:33.242509 (+ 252us) client_negotiation.cc:240] Negotiation successful
0806 16:21:33.242978 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":211,"threads_started":1}
I20260806 16:21:33.245973 5159 heartbeater.cc:347] Connected to a master server at 127.11.245.190:35599
I20260806 16:21:33.246462 5159 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:33.247294 5159 heartbeater.cc:511] Master 127.11.245.190:35599 requested a full tablet report, sending...
I20260806 16:21:33.248904 4523 ts_manager.cc:194] Registered new tserver with Master: 6214b14b98cc42dda87a62e625f31d09 (127.11.245.131:35391)
I20260806 16:21:33.249660 4523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:35179
I20260806 16:21:33.252769 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:21:33.266841 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:33.255460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52494 (local address 127.11.245.190:35599)
0806 16:21:33.255668 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:33.255672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:33.255786 (+ 114us) server_negotiation.cc:407] Connection header received
0806 16:21:33.255886 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:33.255888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:33.255943 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:33.256041 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:33.257338 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.258133 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.259549 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.259809 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.259987 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:33.261954 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:33.261960 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:33.261968 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:33.262002 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:33.264251 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:33.265268 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:33.265272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:33.265274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:33.265376 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:33.265722 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:33.265726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:33.265729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:33.266052 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:33.266220 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:33.266487 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 16:21:33.266636 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 16:21:33.271219 4523 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:21:33.275458 4523 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:21:33.295854 5179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:33.288436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:40925 (local address 127.0.0.1:40944)
0806 16:21:33.289676 (+ 1240us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:33.289693 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:33.289799 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:33.290266 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:33.290274 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:33.290316 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:33.290561 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:33.290571 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.292539 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.292543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:33.293813 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:33.293821 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.295293 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.295300 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:33.295396 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:33.295420 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 16:21:33.295492 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 16:21:33.295631 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1099,"mutex_wait_us":183,"spinlock_wait_cycles":7424,"thread_start_us":105,"threads_started":1}
I20260806 16:21:33.296216 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:33.289021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48198 (local address 127.11.245.131:35391)
0806 16:21:33.289650 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:33.289670 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:33.289701 (+ 31us) server_negotiation.cc:407] Connection header received
0806 16:21:33.289808 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:33.289814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:33.290096 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:33.290286 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:33.290729 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.291753 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.293864 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.294847 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.295023 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:33.295207 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:33.295716 (+ 509us) server_negotiation.cc:299] Negotiation successful
0806 16:21:33.295872 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":117,"threads_started":1}
I20260806 16:21:33.296473 5178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:33.287932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:35391 (local address 127.0.0.1:48198)
0806 16:21:33.289173 (+ 1241us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:33.289242 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:33.289413 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:33.290267 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:33.290274 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:33.290319 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:33.290576 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:33.290584 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.291957 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.291961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:33.293569 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:33.293581 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.295020 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.295025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:33.295162 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:33.295218 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 16:21:33.295389 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 16:21:33.296320 (+ 931us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":978,"mutex_wait_us":777,"spinlock_wait_cycles":5248,"thread_start_us":187,"threads_started":1}
I20260806 16:21:33.296929 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:33.288678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40944 (local address 127.11.245.130:40925)
0806 16:21:33.289793 (+ 1115us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:33.289807 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:33.289834 (+ 27us) server_negotiation.cc:407] Connection header received
0806 16:21:33.289922 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:33.289928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:33.290089 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:33.290269 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:33.290664 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.291836 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.293948 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.294856 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.295013 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:33.295214 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:33.296665 (+ 1451us) server_negotiation.cc:299] Negotiation successful
0806 16:21:33.296797 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":984,"thread_start_us":121,"threads_started":1}
I20260806 16:21:33.297232 5180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:33.288175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:37499 (local address 127.0.0.1:34912)
0806 16:21:33.289172 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:33.289243 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:33.290488 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:33.291060 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:33.291063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:33.291082 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:33.291489 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:33.291501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.292861 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.292865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:33.294182 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:33.294196 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.295597 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.295602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:33.295716 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:33.295741 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 16:21:33.295814 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 16:21:33.295884 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":876,"mutex_wait_us":1,"thread_start_us":139,"threads_started":1}
I20260806 16:21:33.298002 5182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:33.288920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34912 (local address 127.11.245.129:37499)
0806 16:21:33.289906 (+ 986us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:33.289913 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:33.289936 (+ 23us) server_negotiation.cc:407] Connection header received
0806 16:21:33.290669 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:33.290678 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:33.290896 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:33.291073 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:33.291634 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.292710 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.294348 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.295242 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.295378 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:33.295534 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:33.295970 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 16:21:33.297828 (+ 1858us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":867,"thread_start_us":120,"threads_started":1}
I20260806 16:21:33.300704 4890 tablet_service.cc:1467] Processing CreateTablet for tablet bc37dc358ca049918cf13c0e0c71467f (DEFAULT_TABLE table=test-table [id=e16e488594944d8ca2f9c1745f476d57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:33.300702 5072 tablet_service.cc:1467] Processing CreateTablet for tablet bc37dc358ca049918cf13c0e0c71467f (DEFAULT_TABLE table=test-table [id=e16e488594944d8ca2f9c1745f476d57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:33.301103 4700 tablet_service.cc:1467] Processing CreateTablet for tablet bc37dc358ca049918cf13c0e0c71467f (DEFAULT_TABLE table=test-table [id=e16e488594944d8ca2f9c1745f476d57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:33.302210 4890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc37dc358ca049918cf13c0e0c71467f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:33.302212 5072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc37dc358ca049918cf13c0e0c71467f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:33.302213 4700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc37dc358ca049918cf13c0e0c71467f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:33.312448 5185 tablet_bootstrap.cc:492] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223: Bootstrap starting.
I20260806 16:21:33.312709 5186 tablet_bootstrap.cc:492] T bc37dc358ca049918cf13c0e0c71467f P 3ae86c82305d4a6ab1cd9f2b8bcc225f: Bootstrap starting.
I20260806 16:21:33.315340 5186 tablet_bootstrap.cc:654] T bc37dc358ca049918cf13c0e0c71467f P 3ae86c82305d4a6ab1cd9f2b8bcc225f: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:33.315339 5185 tablet_bootstrap.cc:654] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:33.316694 5187 tablet_bootstrap.cc:492] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09: Bootstrap starting.
I20260806 16:21:33.316895 5186 log.cc:824] T bc37dc358ca049918cf13c0e0c71467f P 3ae86c82305d4a6ab1cd9f2b8bcc225f: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:33.317260 5185 log.cc:824] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:33.319938 5187 tablet_bootstrap.cc:654] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:33.321206 5185 tablet_bootstrap.cc:492] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223: No bootstrap required, opened a new log
I20260806 16:21:33.321208 5186 tablet_bootstrap.cc:492] T bc37dc358ca049918cf13c0e0c71467f P 3ae86c82305d4a6ab1cd9f2b8bcc225f: No bootstrap required, opened a new log
I20260806 16:21:33.321501 5185 ts_tablet_manager.cc:1397] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260806 16:21:33.321568 5186 ts_tablet_manager.cc:1397] T bc37dc358ca049918cf13c0e0c71467f P 3ae86c82305d4a6ab1cd9f2b8bcc225f: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 16:21:33.321660 5187 log.cc:824] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:33.324692 5187 tablet_bootstrap.cc:492] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09: No bootstrap required, opened a new log
I20260806 16:21:33.325032 5187 ts_tablet_manager.cc:1397] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 16:21:33.327087 5185 raft_consensus.cc:348] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6214b14b98cc42dda87a62e625f31d09" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35391 } } peers { permanent_uuid: "3ae86c82305d4a6ab1cd9f2b8bcc225f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37499 } } peers { permanent_uuid: "35d85934c6134bea97f11f32619ed223" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40925 } }
I20260806 16:21:33.327491 5185 raft_consensus.cc:374] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:33.327548 5185 raft_consensus.cc:729] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35d85934c6134bea97f11f32619ed223, State: Initialized, Role: FOLLOWER
I20260806 16:21:33.327332 5186 raft_consensus.cc:348] T bc37dc358ca049918cf13c0e0c71467f P 3ae86c82305d4a6ab1cd9f2b8bcc225f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6214b14b98cc42dda87a62e625f31d09" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35391 } } peers { permanent_uuid: "3ae86c82305d4a6ab1cd9f2b8bcc225f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37499 } } peers { permanent_uuid: "35d85934c6134bea97f11f32619ed223" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40925 } }
I20260806 16:21:33.327625 5186 raft_consensus.cc:374] T bc37dc358ca049918cf13c0e0c71467f P 3ae86c82305d4a6ab1cd9f2b8bcc225f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:33.327684 5186 raft_consensus.cc:729] T bc37dc358ca049918cf13c0e0c71467f P 3ae86c82305d4a6ab1cd9f2b8bcc225f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae86c82305d4a6ab1cd9f2b8bcc225f, State: Initialized, Role: FOLLOWER
I20260806 16:21:33.328222 5186 consensus_queue.cc:260] T bc37dc358ca049918cf13c0e0c71467f P 3ae86c82305d4a6ab1cd9f2b8bcc225f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6214b14b98cc42dda87a62e625f31d09" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35391 } } peers { permanent_uuid: "3ae86c82305d4a6ab1cd9f2b8bcc225f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37499 } } peers { permanent_uuid: "35d85934c6134bea97f11f32619ed223" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40925 } }
I20260806 16:21:33.328220 5185 consensus_queue.cc:260] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6214b14b98cc42dda87a62e625f31d09" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35391 } } peers { permanent_uuid: "3ae86c82305d4a6ab1cd9f2b8bcc225f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37499 } } peers { permanent_uuid: "35d85934c6134bea97f11f32619ed223" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40925 } }
I20260806 16:21:33.329185 5187 raft_consensus.cc:348] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6214b14b98cc42dda87a62e625f31d09" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35391 } } peers { permanent_uuid: "3ae86c82305d4a6ab1cd9f2b8bcc225f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37499 } } peers { permanent_uuid: "35d85934c6134bea97f11f32619ed223" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40925 } }
I20260806 16:21:33.329479 5187 raft_consensus.cc:374] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:33.329556 5187 raft_consensus.cc:729] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6214b14b98cc42dda87a62e625f31d09, State: Initialized, Role: FOLLOWER
I20260806 16:21:33.329608 5186 ts_tablet_manager.cc:1428] T bc37dc358ca049918cf13c0e0c71467f P 3ae86c82305d4a6ab1cd9f2b8bcc225f: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 16:21:33.329607 5185 ts_tablet_manager.cc:1428] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260806 16:21:33.330091 4794 heartbeater.cc:503] Master 127.11.245.190:35599 was elected leader, sending a full tablet report...
I20260806 16:21:33.330091 4972 heartbeater.cc:503] Master 127.11.245.190:35599 was elected leader, sending a full tablet report...
I20260806 16:21:33.330209 5187 consensus_queue.cc:260] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6214b14b98cc42dda87a62e625f31d09" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35391 } } peers { permanent_uuid: "3ae86c82305d4a6ab1cd9f2b8bcc225f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37499 } } peers { permanent_uuid: "35d85934c6134bea97f11f32619ed223" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40925 } }
I20260806 16:21:33.331238 5159 heartbeater.cc:503] Master 127.11.245.190:35599 was elected leader, sending a full tablet report...
I20260806 16:21:33.332191 5187 ts_tablet_manager.cc:1428] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09: Time spent starting tablet: real 0.007s user 0.003s sys 0.005s
W20260806 16:21:33.405050 4975 tablet.cc:2369] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:21:33.454766 5199 raft_consensus.cc:482] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:21:33.455082 5199 raft_consensus.cc:504] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6214b14b98cc42dda87a62e625f31d09" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35391 } } peers { permanent_uuid: "3ae86c82305d4a6ab1cd9f2b8bcc225f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37499 } } peers { permanent_uuid: "35d85934c6134bea97f11f32619ed223" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40925 } }
I20260806 16:21:33.456859 5199 leader_election.cc:288] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae86c82305d4a6ab1cd9f2b8bcc225f (127.11.245.129:37499), 35d85934c6134bea97f11f32619ed223 (127.11.245.130:40925)
I20260806 16:21:33.461920 5182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:33.457468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:55711 (local address 127.11.245.129:37499)
0806 16:21:33.457630 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:33.457635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:33.457659 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:21:33.457736 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:33.457740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:33.457804 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:33.457903 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:33.458288 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.459081 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.460578 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.461083 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.461243 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:33.461332 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:33.461662 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 16:21:33.461775 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 16:21:33.461920 5162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:33.457227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:37499 (local address 127.11.245.131:55711)
0806 16:21:33.457431 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:33.457455 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:33.457590 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:33.457931 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:33.457934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:33.457958 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:33.458156 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:33.458163 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.459242 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.459247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:33.460399 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:33.460412 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.461238 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.461243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:33.461458 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:33.461485 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 16:21:33.461595 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 16:21:33.461704 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":58}
I20260806 16:21:33.461926 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:33.457502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:52257 (local address 127.11.245.130:40925)
0806 16:21:33.457742 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:33.457750 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:33.457924 (+ 174us) server_negotiation.cc:407] Connection header received
0806 16:21:33.458085 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:33.458088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:33.458139 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:33.458211 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:33.458619 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.459133 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.460699 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.461147 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.461285 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:33.461355 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:33.461658 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 16:21:33.461755 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260806 16:21:33.461931 5201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:33.457264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:40925 (local address 127.11.245.131:52257)
0806 16:21:33.457773 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:33.457786 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:33.457931 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:33.458362 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:33.458364 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:33.458379 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:33.458501 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:33.458505 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.459252 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.459257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:33.460557 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:33.460569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.461286 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.461290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:33.461429 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:33.461496 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 16:21:33.461633 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 16:21:33.461707 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":152,"thread_start_us":183,"threads_started":1}
I20260806 16:21:33.462988 4725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc37dc358ca049918cf13c0e0c71467f" candidate_uuid: "6214b14b98cc42dda87a62e625f31d09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae86c82305d4a6ab1cd9f2b8bcc225f" is_pre_election: true
I20260806 16:21:33.462982 4913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc37dc358ca049918cf13c0e0c71467f" candidate_uuid: "6214b14b98cc42dda87a62e625f31d09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35d85934c6134bea97f11f32619ed223" is_pre_election: true
I20260806 16:21:33.463488 4913 raft_consensus.cc:2454] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6214b14b98cc42dda87a62e625f31d09 in term 0.
I20260806 16:21:33.463488 4725 raft_consensus.cc:2454] T bc37dc358ca049918cf13c0e0c71467f P 3ae86c82305d4a6ab1cd9f2b8bcc225f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6214b14b98cc42dda87a62e625f31d09 in term 0.
I20260806 16:21:33.464349 5004 leader_election.cc:302] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 35d85934c6134bea97f11f32619ed223, 3ae86c82305d4a6ab1cd9f2b8bcc225f, 6214b14b98cc42dda87a62e625f31d09; no voters:
I20260806 16:21:33.465046 5199 raft_consensus.cc:2781] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:21:33.465183 5199 raft_consensus.cc:482] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:21:33.465286 5199 raft_consensus.cc:3037] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:33.466773 5199 raft_consensus.cc:504] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6214b14b98cc42dda87a62e625f31d09" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35391 } } peers { permanent_uuid: "3ae86c82305d4a6ab1cd9f2b8bcc225f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37499 } } peers { permanent_uuid: "35d85934c6134bea97f11f32619ed223" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40925 } }
I20260806 16:21:33.467303 5199 leader_election.cc:288] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [CANDIDATE]: Term 1 election: Requested vote from peers 3ae86c82305d4a6ab1cd9f2b8bcc225f (127.11.245.129:37499), 35d85934c6134bea97f11f32619ed223 (127.11.245.130:40925)
I20260806 16:21:33.468130 4913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc37dc358ca049918cf13c0e0c71467f" candidate_uuid: "6214b14b98cc42dda87a62e625f31d09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35d85934c6134bea97f11f32619ed223"
I20260806 16:21:33.468130 4725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc37dc358ca049918cf13c0e0c71467f" candidate_uuid: "6214b14b98cc42dda87a62e625f31d09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae86c82305d4a6ab1cd9f2b8bcc225f"
I20260806 16:21:33.468324 4913 raft_consensus.cc:3037] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:33.468365 4725 raft_consensus.cc:3037] T bc37dc358ca049918cf13c0e0c71467f P 3ae86c82305d4a6ab1cd9f2b8bcc225f [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:33.470095 4913 raft_consensus.cc:2454] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6214b14b98cc42dda87a62e625f31d09 in term 1.
I20260806 16:21:33.470095 4725 raft_consensus.cc:2454] T bc37dc358ca049918cf13c0e0c71467f P 3ae86c82305d4a6ab1cd9f2b8bcc225f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6214b14b98cc42dda87a62e625f31d09 in term 1.
I20260806 16:21:33.470615 5004 leader_election.cc:302] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35d85934c6134bea97f11f32619ed223, 6214b14b98cc42dda87a62e625f31d09; no voters:
I20260806 16:21:33.470928 5199 raft_consensus.cc:2781] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:33.471266 5199 raft_consensus.cc:686] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [term 1 LEADER]: Becoming Leader. State: Replica: 6214b14b98cc42dda87a62e625f31d09, State: Running, Role: LEADER
I20260806 16:21:33.471761 5199 consensus_queue.cc:237] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6214b14b98cc42dda87a62e625f31d09" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35391 } } peers { permanent_uuid: "3ae86c82305d4a6ab1cd9f2b8bcc225f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37499 } } peers { permanent_uuid: "35d85934c6134bea97f11f32619ed223" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40925 } }
W20260806 16:21:33.473747 5160 tablet.cc:2369] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:21:33.476658 4520 catalog_manager.cc:5495] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6214b14b98cc42dda87a62e625f31d09 (127.11.245.131). New cstate: current_term: 1 leader_uuid: "6214b14b98cc42dda87a62e625f31d09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6214b14b98cc42dda87a62e625f31d09" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 35391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ae86c82305d4a6ab1cd9f2b8bcc225f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35d85934c6134bea97f11f32619ed223" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40925 } health_report { overall_health: UNKNOWN } } }
I20260806 16:21:33.494287 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:33.489610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48204 (local address 127.11.245.131:35391)
0806 16:21:33.489758 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:33.489763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:33.489776 (+ 13us) server_negotiation.cc:407] Connection header received
0806 16:21:33.489883 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:33.489886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:33.489956 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:33.490068 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:21:33.490471 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.491033 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.492276 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.492532 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.492634 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:33.493081 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:21:33.493749 (+ 668us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:21:33.493874 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:33.493990 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 16:21:33.494073 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 16:21:33.503192 4725 raft_consensus.cc:1261] T bc37dc358ca049918cf13c0e0c71467f P 3ae86c82305d4a6ab1cd9f2b8bcc225f [term 1 FOLLOWER]: Refusing update from remote peer 6214b14b98cc42dda87a62e625f31d09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:21:33.503191 4913 raft_consensus.cc:1261] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223 [term 1 FOLLOWER]: Refusing update from remote peer 6214b14b98cc42dda87a62e625f31d09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:21:33.504477 5204 consensus_queue.cc:1035] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae86c82305d4a6ab1cd9f2b8bcc225f" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:21:33.504789 5199 consensus_queue.cc:1035] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35d85934c6134bea97f11f32619ed223" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:21:33.517685 5209 mvcc.cc:205] Tried to move back new op lower bound from 7315592370177593344 to 7315592370063990784. Current Snapshot: MvccSnapshot[applied={T|T < 7315592370177593344}]
I20260806 16:21:33.527469 4520 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:21:33.527789 4520 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:21:33.531682 4520 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b82cf6fd3774c98a19bba28d0ce3247: Sending DeleteTablet for 3 replicas of tablet bc37dc358ca049918cf13c0e0c71467f
I20260806 16:21:33.533118 4890 tablet_service.cc:1514] Processing DeleteTablet for tablet bc37dc358ca049918cf13c0e0c71467f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:33 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:40944
I20260806 16:21:33.533092 5072 tablet_service.cc:1514] Processing DeleteTablet for tablet bc37dc358ca049918cf13c0e0c71467f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:33 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:48198
I20260806 16:21:33.533090 4700 tablet_service.cc:1514] Processing DeleteTablet for tablet bc37dc358ca049918cf13c0e0c71467f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:33 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:34912
I20260806 16:21:33.534133 5228 tablet_replica.cc:330] T bc37dc358ca049918cf13c0e0c71467f P 3ae86c82305d4a6ab1cd9f2b8bcc225f: stopping tablet replica
I20260806 16:21:33.534660 5228 raft_consensus.cc:2229] T bc37dc358ca049918cf13c0e0c71467f P 3ae86c82305d4a6ab1cd9f2b8bcc225f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:21:33.535156 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 4613
I20260806 16:21:33.542796 5227 tablet_replica.cc:330] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223: stopping tablet replica
I20260806 16:21:33.543371 5227 raft_consensus.cc:2229] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:21:33.544130 5227 raft_consensus.cc:2258] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:21:33.547441 5225 tablet_replica.cc:330] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09: stopping tablet replica
I20260806 16:21:33.548020 5225 raft_consensus.cc:2229] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:21:33.548708 5225 raft_consensus.cc:2258] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:21:33.549821 5227 ts_tablet_manager.cc:1905] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:21:33.550625 5225 ts_tablet_manager.cc:1905] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:21:33.554539 5227 ts_tablet_manager.cc:1918] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:21:33.554414 5225 ts_tablet_manager.cc:1918] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:21:33.554790 5225 log.cc:1196] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/bc37dc358ca049918cf13c0e0c71467f
I20260806 16:21:33.554961 5227 log.cc:1196] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/bc37dc358ca049918cf13c0e0c71467f
I20260806 16:21:33.555274 5225 ts_tablet_manager.cc:1939] T bc37dc358ca049918cf13c0e0c71467f P 6214b14b98cc42dda87a62e625f31d09: Deleting consensus metadata
I20260806 16:21:33.555693 5227 ts_tablet_manager.cc:1939] T bc37dc358ca049918cf13c0e0c71467f P 35d85934c6134bea97f11f32619ed223: Deleting consensus metadata
I20260806 16:21:33.557467 4503 catalog_manager.cc:4841] TS 6214b14b98cc42dda87a62e625f31d09 (127.11.245.131:35391): tablet bc37dc358ca049918cf13c0e0c71467f (table test-table [id=e16e488594944d8ca2f9c1745f476d57]) successfully deleted
I20260806 16:21:33.558188 4504 catalog_manager.cc:4841] TS 35d85934c6134bea97f11f32619ed223 (127.11.245.130:40925): tablet bc37dc358ca049918cf13c0e0c71467f (table test-table [id=e16e488594944d8ca2f9c1745f476d57]) successfully deleted
W20260806 16:21:33.559542 4502 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:37499 (error 108)
I20260806 16:21:33.560858 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 4799
I20260806 16:21:33.561190 5179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:33.560669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:37499 (local address 127.0.0.1:34918)
0806 16:21:33.560957 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:33.561058 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:37499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
W20260806 16:21:33.580696 4502 catalog_manager.cc:4584] TS 3ae86c82305d4a6ab1cd9f2b8bcc225f (127.11.245.129:37499): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bc37dc358ca049918cf13c0e0c71467f: Network error: Client connection negotiation failed: client connection to 127.11.245.129:37499: connect: Connection refused (error 111)
I20260806 16:21:33.587857 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 4979
I20260806 16:21:33.607676 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 4481
2026-08-06T16:21:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (3159 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T16:21:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:21:33Z Disabled control of system clock
I20260806 16:21:33.661885 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:34061
--webserver_interface=127.11.245.190
--webserver_port=0
--builtin_ntp_servers=127.11.245.148:37799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:34061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:21:33.816394 5234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:33.816862 5234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:33.816951 5234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:33.821806 5234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:21:33.821954 5234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:33.821987 5234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:33.822285 5234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:21:33.822371 5234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:21:33.828289 5234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:34061
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:34061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:33.830034 5234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:33.832013 5234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:33.844147 5245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:33.844147 5240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:33.844172 5241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:33.844688 5234 server_base.cc:1034] running on GCE node
I20260806 16:21:33.845525 5234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:33.847330 5234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:33.848560 5234 hybrid_clock.cc:648] HybridClock initialized: now 1786033293848534 us; error 49 us; skew 500 ppm
I20260806 16:21:33.851385 5234 webserver.cc:460] Webserver started at http://127.11.245.190:43607/ using document root <none> and password file <none>
I20260806 16:21:33.852311 5234 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:33.852425 5234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:33.852741 5234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:33.855567 5234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "539a5939ad13443795c0f025f6005628"
format_stamp: "Formatted at 2026-08-06 16:21:33 on dist-test-slave-t3q3"
server_key: "fa7798370b96f469337161d9fe134c89"
server_key_iv: "da9e20deae9d99f85a8999b0ab359e90"
server_key_version: "encryptionkey@0"
I20260806 16:21:33.856395 5234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "539a5939ad13443795c0f025f6005628"
format_stamp: "Formatted at 2026-08-06 16:21:33 on dist-test-slave-t3q3"
server_key: "fa7798370b96f469337161d9fe134c89"
server_key_iv: "da9e20deae9d99f85a8999b0ab359e90"
server_key_version: "encryptionkey@0"
I20260806 16:21:33.861987 5234 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260806 16:21:33.865715 5257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:33.867527 5234 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:21:33.867731 5234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "539a5939ad13443795c0f025f6005628"
format_stamp: "Formatted at 2026-08-06 16:21:33 on dist-test-slave-t3q3"
server_key: "fa7798370b96f469337161d9fe134c89"
server_key_iv: "da9e20deae9d99f85a8999b0ab359e90"
server_key_version: "encryptionkey@0"
I20260806 16:21:33.867906 5234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:33.887672 5234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:33.888731 5234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:33.888994 5234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:33.899281 5234 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:34061
I20260806 16:21:33.899333 5318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:34061 every 8 connection(s)
I20260806 16:21:33.900792 5234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:21:33.903854 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 5234
I20260806 16:21:33.904173 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:21:33.904629 12246 external_mini_cluster.cc:1428] Setting key d05db21d21bcde43195b4bf3d43966a3
I20260806 16:21:33.907588 5319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:33.916010 5319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628: Bootstrap starting.
I20260806 16:21:33.918678 5322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:33.906948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51098 (local address 127.11.245.190:34061)
0806 16:21:33.908507 (+ 1559us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:33.908534 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:33.908596 (+ 62us) server_negotiation.cc:407] Connection header received
0806 16:21:33.909712 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:33.909741 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:33.910356 (+ 615us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:33.910925 (+ 569us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:33.911565 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.912890 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.914047 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:33.914649 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:33.914859 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:33.916107 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:33.916118 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:33.916140 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:33.916785 (+ 645us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:33.916859 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:33.916888 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:33.917024 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:33.917198 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 16:21:33.917559 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":236,"threads_started":1}
I20260806 16:21:33.919422 5319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:33.920603 5319 log.cc:824] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:33.923365 5319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628: No bootstrap required, opened a new log
I20260806 16:21:33.928957 5319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539a5939ad13443795c0f025f6005628" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34061 } }
I20260806 16:21:33.929342 5319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:33.929382 5319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 539a5939ad13443795c0f025f6005628, State: Initialized, Role: FOLLOWER
I20260806 16:21:33.929952 5319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539a5939ad13443795c0f025f6005628" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34061 } }
I20260806 16:21:33.930111 5319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:21:33.930181 5319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:21:33.930274 5319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:33.931514 5319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539a5939ad13443795c0f025f6005628" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34061 } }
I20260806 16:21:33.931937 5319 leader_election.cc:302] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 539a5939ad13443795c0f025f6005628; no voters:
I20260806 16:21:33.932358 5319 leader_election.cc:288] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:21:33.932714 5324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:33.933084 5324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [term 1 LEADER]: Becoming Leader. State: Replica: 539a5939ad13443795c0f025f6005628, State: Running, Role: LEADER
I20260806 16:21:33.933492 5324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539a5939ad13443795c0f025f6005628" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34061 } }
I20260806 16:21:33.934371 5319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:21:33.935588 5325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "539a5939ad13443795c0f025f6005628" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539a5939ad13443795c0f025f6005628" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34061 } } }
I20260806 16:21:33.935743 5325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:33.936262 5336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:21:33.936601 5326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 539a5939ad13443795c0f025f6005628. Latest consensus state: current_term: 1 leader_uuid: "539a5939ad13443795c0f025f6005628" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539a5939ad13443795c0f025f6005628" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34061 } } }
I20260806 16:21:33.936792 5326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628 [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:33.940502 5336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:21:33.948726 5336 catalog_manager.cc:1347] Generated new cluster ID: 568b28a6e2d8433a96ce368159195d77
I20260806 16:21:33.948848 5336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:21:33.968683 5336 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:21:33.970310 5336 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:21:33.978890 5336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628: Generated new TSK 0
I20260806 16:21:33.980458 5336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:21:33.988983 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34061
--builtin_ntp_servers=127.11.245.148:37799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:21:34.152689 5351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:34.153151 5351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:34.153204 5351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:34.158605 5351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:34.158705 5351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:34.158777 5351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:21:34.164701 5351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:34.166466 5351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:34.168789 5351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:34.182680 5359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:34.182667 5366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:34.182832 5351 server_base.cc:1034] running on GCE node
W20260806 16:21:34.182678 5360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:34.183791 5351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:34.184777 5351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:34.186043 5351 hybrid_clock.cc:648] HybridClock initialized: now 1786033294185996 us; error 63 us; skew 500 ppm
I20260806 16:21:34.194396 5351 webserver.cc:460] Webserver started at http://127.11.245.129:37185/ using document root <none> and password file <none>
I20260806 16:21:34.195487 5351 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:34.195873 5351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:34.196317 5351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:34.199288 5351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "da343fcf2bf447da8fe7d08885614e50"
format_stamp: "Formatted at 2026-08-06 16:21:34 on dist-test-slave-t3q3"
server_key: "afe823ea75a8e5813d0a83383ae368dc"
server_key_iv: "3271304f5e5f9423db5af6327c4ba091"
server_key_version: "encryptionkey@0"
I20260806 16:21:34.200439 5351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "da343fcf2bf447da8fe7d08885614e50"
format_stamp: "Formatted at 2026-08-06 16:21:34 on dist-test-slave-t3q3"
server_key: "afe823ea75a8e5813d0a83383ae368dc"
server_key_iv: "3271304f5e5f9423db5af6327c4ba091"
server_key_version: "encryptionkey@0"
I20260806 16:21:34.206136 5351 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:21:34.210029 5377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:34.212020 5351 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:21:34.212306 5351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "da343fcf2bf447da8fe7d08885614e50"
format_stamp: "Formatted at 2026-08-06 16:21:34 on dist-test-slave-t3q3"
server_key: "afe823ea75a8e5813d0a83383ae368dc"
server_key_iv: "3271304f5e5f9423db5af6327c4ba091"
server_key_version: "encryptionkey@0"
I20260806 16:21:34.212512 5351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:34.238060 5351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:34.239244 5351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:34.239518 5351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:34.240514 5351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:34.241911 5351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:34.241990 5351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:34.242097 5351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:34.242139 5351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:34.254171 5351 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:33057
I20260806 16:21:34.254266 5539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:33057 every 8 connection(s)
I20260806 16:21:34.255486 5351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:21:34.261780 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 5351
I20260806 16:21:34.262151 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:21:34.262590 12246 external_mini_cluster.cc:1428] Setting key 85c209c05f82cfab1720a91210c942f6
I20260806 16:21:34.265424 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34061
--builtin_ntp_servers=127.11.245.148:37799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:21:34.269701 5322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.258155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:48377 (local address 127.11.245.190:34061)
0806 16:21:34.258468 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:34.258475 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:34.259944 (+ 1469us) server_negotiation.cc:407] Connection header received
0806 16:21:34.261859 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:34.261864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:34.261953 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:34.262076 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:34.263515 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.264452 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.266661 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.266919 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.267027 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.268906 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:34.268913 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:34.268918 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:34.268943 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:34.268980 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:34.268992 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:34.269094 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:34.269320 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 16:21:34.269444 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172}
I20260806 16:21:34.270875 5543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.258777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:34061 (local address 127.11.245.129:48377)
0806 16:21:34.259719 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:34.259772 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:34.261519 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:34.262373 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:34.262382 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:34.262502 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:34.263282 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:34.263308 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.264637 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.264641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:34.266455 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:34.266477 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.267142 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.267149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:34.267457 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.268496 (+ 1039us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:21:34.268498 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:34.268563 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:21:34.268670 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:34.269121 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:34.269133 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:34.269140 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 16:21:34.269306 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 16:21:34.269676 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":163,"threads_started":1}
I20260806 16:21:34.272699 5541 heartbeater.cc:347] Connected to a master server at 127.11.245.190:34061
I20260806 16:21:34.273114 5541 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:34.274151 5541 heartbeater.cc:511] Master 127.11.245.190:34061 requested a full tablet report, sending...
I20260806 16:21:34.276553 5274 ts_manager.cc:194] Registered new tserver with Master: da343fcf2bf447da8fe7d08885614e50 (127.11.245.129:33057)
I20260806 16:21:34.278188 5274 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.245.129:48377
W20260806 16:21:34.422533 5545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:34.422987 5545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:34.423089 5545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:34.427917 5545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:34.428050 5545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:34.428184 5545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:21:34.434443 5545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:34.436340 5545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:34.438522 5545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:34.451205 5554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:34.451514 5545 server_base.cc:1034] running on GCE node
W20260806 16:21:34.451712 5551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:34.451756 5552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:34.452425 5545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:34.453377 5545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:34.454593 5545 hybrid_clock.cc:648] HybridClock initialized: now 1786033294454558 us; error 56 us; skew 500 ppm
I20260806 16:21:34.457219 5545 webserver.cc:460] Webserver started at http://127.11.245.130:38665/ using document root <none> and password file <none>
I20260806 16:21:34.458027 5545 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:34.458134 5545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:34.458434 5545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:34.461223 5545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "816d0a661a904e849b5b3541e9583285"
format_stamp: "Formatted at 2026-08-06 16:21:34 on dist-test-slave-t3q3"
server_key: "3fa4c3551a739369dc8ba8525296ad59"
server_key_iv: "d127ceb590d7197d76b95972a1390092"
server_key_version: "encryptionkey@0"
I20260806 16:21:34.461921 5545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "816d0a661a904e849b5b3541e9583285"
format_stamp: "Formatted at 2026-08-06 16:21:34 on dist-test-slave-t3q3"
server_key: "3fa4c3551a739369dc8ba8525296ad59"
server_key_iv: "d127ceb590d7197d76b95972a1390092"
server_key_version: "encryptionkey@0"
I20260806 16:21:34.466730 5545 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:21:34.470189 5565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:34.472122 5545 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:21:34.472332 5545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "816d0a661a904e849b5b3541e9583285"
format_stamp: "Formatted at 2026-08-06 16:21:34 on dist-test-slave-t3q3"
server_key: "3fa4c3551a739369dc8ba8525296ad59"
server_key_iv: "d127ceb590d7197d76b95972a1390092"
server_key_version: "encryptionkey@0"
I20260806 16:21:34.472512 5545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:34.496968 5545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:34.498137 5545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:34.498391 5545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:34.499199 5545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:34.500542 5545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:34.500619 5545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:34.500705 5545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:34.500759 5545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:34.512300 5545 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:37923
I20260806 16:21:34.512332 5729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:37923 every 8 connection(s)
I20260806 16:21:34.513563 5545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:21:34.518337 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 5545
I20260806 16:21:34.518476 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:21:34.518858 12246 external_mini_cluster.cc:1428] Setting key 158ee97f3059b943f6a1827878bc8773
I20260806 16:21:34.521723 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34061
--builtin_ntp_servers=127.11.245.148:37799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:21:34.527019 5322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.516004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:55801 (local address 127.11.245.190:34061)
0806 16:21:34.516212 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:34.516216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:34.517724 (+ 1508us) server_negotiation.cc:407] Connection header received
0806 16:21:34.519937 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:34.519940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:34.520014 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:34.520162 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:34.521449 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.522130 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.524180 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.524645 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.524789 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.526210 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:34.526218 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:34.526224 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:34.526248 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:34.526279 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:34.526293 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:34.526388 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:34.526658 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 16:21:34.526767 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 16:21:34.528278 5735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.516618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:34061 (local address 127.11.245.130:55801)
0806 16:21:34.517514 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:34.517597 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:34.519668 (+ 2071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:34.520402 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:34.520411 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:34.520524 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:34.521265 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:34.521289 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.522277 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.522284 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:34.523957 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:34.523975 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.524848 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.524851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:34.525040 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.525844 (+ 804us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:21:34.525846 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:34.525902 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:21:34.526000 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:34.526428 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:34.526440 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:34.526447 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 16:21:34.526624 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 16:21:34.526980 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":136,"threads_started":1}
I20260806 16:21:34.530185 5730 heartbeater.cc:347] Connected to a master server at 127.11.245.190:34061
I20260806 16:21:34.530592 5730 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:34.531594 5730 heartbeater.cc:511] Master 127.11.245.190:34061 requested a full tablet report, sending...
I20260806 16:21:34.533411 5274 ts_manager.cc:194] Registered new tserver with Master: 816d0a661a904e849b5b3541e9583285 (127.11.245.130:37923)
I20260806 16:21:34.534209 5274 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.245.130:55801
W20260806 16:21:34.674996 5736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:34.675439 5736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:34.675493 5736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:34.680934 5736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:34.681061 5736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:34.681136 5736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:21:34.687234 5736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:34061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:34.689312 5736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:34.691345 5736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:21:34.705880 5736 server_base.cc:1034] running on GCE node
W20260806 16:21:34.705721 5744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:34.705751 5746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:34.705770 5743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:34.706686 5736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:34.707656 5736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:34.711032 5736 hybrid_clock.cc:648] HybridClock initialized: now 1786033294710912 us; error 123 us; skew 500 ppm
I20260806 16:21:34.714056 5736 webserver.cc:460] Webserver started at http://127.11.245.131:45967/ using document root <none> and password file <none>
I20260806 16:21:34.714910 5736 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:34.715026 5736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:34.715346 5736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:34.718493 5736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "2c8e598e9e4c42dd9436626e64284147"
format_stamp: "Formatted at 2026-08-06 16:21:34 on dist-test-slave-t3q3"
server_key: "54b217c9172ba3f8584ea49e9b75efb1"
server_key_iv: "83174e93617b2b06132d2391301f7e96"
server_key_version: "encryptionkey@0"
I20260806 16:21:34.719575 5736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "2c8e598e9e4c42dd9436626e64284147"
format_stamp: "Formatted at 2026-08-06 16:21:34 on dist-test-slave-t3q3"
server_key: "54b217c9172ba3f8584ea49e9b75efb1"
server_key_iv: "83174e93617b2b06132d2391301f7e96"
server_key_version: "encryptionkey@0"
I20260806 16:21:34.724870 5736 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 16:21:34.728531 5755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:34.730808 5736 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:21:34.731021 5736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "2c8e598e9e4c42dd9436626e64284147"
format_stamp: "Formatted at 2026-08-06 16:21:34 on dist-test-slave-t3q3"
server_key: "54b217c9172ba3f8584ea49e9b75efb1"
server_key_iv: "83174e93617b2b06132d2391301f7e96"
server_key_version: "encryptionkey@0"
I20260806 16:21:34.731242 5736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:34.753230 5736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:34.754268 5736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:34.754482 5736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:34.755259 5736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:34.756836 5736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:34.756916 5736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:34.756987 5736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:34.757018 5736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:34.768786 5736 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:41483
I20260806 16:21:34.768854 5910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:41483 every 8 connection(s)
I20260806 16:21:34.770098 5736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:21:34.774502 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 5736
I20260806 16:21:34.774670 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:21:34.775095 12246 external_mini_cluster.cc:1428] Setting key 7e983de33d0189d272648eb4b15fc59b
I20260806 16:21:34.784723 5322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.773484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:60485 (local address 127.11.245.190:34061)
0806 16:21:34.773758 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:34.773763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:34.775372 (+ 1609us) server_negotiation.cc:407] Connection header received
0806 16:21:34.778147 (+ 2775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:34.778151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:34.778228 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:34.778331 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:34.780098 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.780695 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.782079 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.782317 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.782440 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.783972 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:34.783976 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:34.783979 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:34.783998 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:34.784025 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:34.784034 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:34.784178 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:34.784421 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 16:21:34.784518 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260806 16:21:34.786070 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.774073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:34061 (local address 127.11.245.131:60485)
0806 16:21:34.775130 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:34.775204 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:34.777530 (+ 2326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:34.778746 (+ 1216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:34.778759 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:34.779092 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:34.779870 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:34.779890 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.780856 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.780860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:34.781931 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:34.781943 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.782520 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.782525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:34.782732 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.783568 (+ 836us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:21:34.783570 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:34.783631 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:21:34.783734 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:34.784183 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:34.784196 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:34.784205 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 16:21:34.784410 (+ 205us) client_negotiation.cc:240] Negotiation successful
0806 16:21:34.784851 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":158,"threads_started":1}
I20260806 16:21:34.787668 5913 heartbeater.cc:347] Connected to a master server at 127.11.245.190:34061
I20260806 16:21:34.788048 5913 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:34.788926 5913 heartbeater.cc:511] Master 127.11.245.190:34061 requested a full tablet report, sending...
I20260806 16:21:34.790472 5271 ts_manager.cc:194] Registered new tserver with Master: 2c8e598e9e4c42dd9436626e64284147 (127.11.245.131:41483)
I20260806 16:21:34.791256 5271 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.245.131:60485
I20260806 16:21:34.804036 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:21:34.813560 5322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.807249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51102 (local address 127.11.245.190:34061)
0806 16:21:34.807476 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:34.807483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:34.807670 (+ 187us) server_negotiation.cc:407] Connection header received
0806 16:21:34.807899 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:34.807902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:34.807965 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:34.808101 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:34.808506 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.809582 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.811096 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.811353 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.811458 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.812827 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:34.812831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:34.812834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:34.812855 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:34.812887 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:34.812898 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:34.812992 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:34.813214 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 16:21:34.813301 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 16:21:34.819370 5271 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:21:34.823814 5271 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:21:34.849802 5935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.841475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58738 (local address 127.11.245.131:41483)
0806 16:21:34.842788 (+ 1313us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:34.842799 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:34.843006 (+ 207us) server_negotiation.cc:407] Connection header received
0806 16:21:34.843269 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:34.843275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:34.843505 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:34.843651 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:34.844121 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.845224 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.847182 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.848261 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.848418 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.848646 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:34.849367 (+ 721us) server_negotiation.cc:299] Negotiation successful
0806 16:21:34.849541 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1213,"thread_start_us":924,"threads_started":1}
I20260806 16:21:34.850390 5926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.839514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:33057 (local address 127.0.0.1:55012)
0806 16:21:34.840161 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:34.840236 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:34.840428 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:34.843215 (+ 2787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:34.843224 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:34.843251 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:34.843450 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:34.843458 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.845167 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.845173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:34.846991 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:34.847005 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.849921 (+ 2916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.849925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:34.850029 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.850053 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 16:21:34.850130 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 16:21:34.850218 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":240,"thread_start_us":137,"threads_started":1}
I20260806 16:21:34.850585 5928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.839955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55012 (local address 127.11.245.129:33057)
0806 16:21:34.842522 (+ 2567us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:34.842531 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:34.842567 (+ 36us) server_negotiation.cc:407] Connection header received
0806 16:21:34.842687 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:34.842694 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:34.843020 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:34.843217 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:34.843605 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.845000 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.847187 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.849491 (+ 2304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.849650 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.849855 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:34.850233 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 16:21:34.850421 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2380,"thread_start_us":210,"threads_started":1}
I20260806 16:21:34.852933 5931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.841021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:37923 (local address 127.0.0.1:52354)
0806 16:21:34.842941 (+ 1920us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:34.842956 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:34.843080 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:34.843639 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:34.843642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:34.843661 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:34.843772 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:34.843782 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.845200 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.845203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:34.846384 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:34.846396 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.848679 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.848683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:34.848909 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.848967 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 16:21:34.849110 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 16:21:34.852585 (+ 3475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1845,"mutex_wait_us":757,"thread_start_us":117,"threads_started":1}
I20260806 16:21:34.853909 5932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.841846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52354 (local address 127.11.245.130:37923)
0806 16:21:34.842551 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:34.842567 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:34.843081 (+ 514us) server_negotiation.cc:407] Connection header received
0806 16:21:34.843267 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:34.843273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:34.843474 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:34.843669 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:34.843881 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.845070 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.846535 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.847883 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.848298 (+ 415us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.848608 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:34.853493 (+ 4885us) server_negotiation.cc:299] Negotiation successful
0806 16:21:34.853694 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":179,"threads_started":1}
I20260806 16:21:34.854282 5929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.841021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:41483 (local address 127.0.0.1:58738)
0806 16:21:34.842868 (+ 1847us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:34.842887 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:34.843098 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:34.843845 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:34.843851 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:34.843872 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:34.843965 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:34.843969 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.845376 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.845379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:34.846991 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:34.847005 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.848717 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.848722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:34.848924 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.848971 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 16:21:34.849113 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 16:21:34.854115 (+ 5002us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1766,"mutex_wait_us":5055,"thread_start_us":164,"threads_started":1}
I20260806 16:21:34.856133 5440 tablet_service.cc:1467] Processing CreateTablet for tablet e4576eacab0140f1bb0b73bfe801d737 (DEFAULT_TABLE table=test-table [id=4ebc30b692b4470e88e48963f4d3bea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:34.857928 5440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4576eacab0140f1bb0b73bfe801d737. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:34.858556 5816 tablet_service.cc:1467] Processing CreateTablet for tablet e4576eacab0140f1bb0b73bfe801d737 (DEFAULT_TABLE table=test-table [id=4ebc30b692b4470e88e48963f4d3bea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:34.858551 5637 tablet_service.cc:1467] Processing CreateTablet for tablet e4576eacab0140f1bb0b73bfe801d737 (DEFAULT_TABLE table=test-table [id=4ebc30b692b4470e88e48963f4d3bea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:34.860019 5637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4576eacab0140f1bb0b73bfe801d737. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:34.860023 5816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4576eacab0140f1bb0b73bfe801d737. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:34.869822 5936 tablet_bootstrap.cc:492] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285: Bootstrap starting.
I20260806 16:21:34.871922 5937 tablet_bootstrap.cc:492] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147: Bootstrap starting.
I20260806 16:21:34.872766 5939 tablet_bootstrap.cc:492] T e4576eacab0140f1bb0b73bfe801d737 P da343fcf2bf447da8fe7d08885614e50: Bootstrap starting.
I20260806 16:21:34.874433 5937 tablet_bootstrap.cc:654] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:34.874502 5936 tablet_bootstrap.cc:654] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:34.875401 5939 tablet_bootstrap.cc:654] T e4576eacab0140f1bb0b73bfe801d737 P da343fcf2bf447da8fe7d08885614e50: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:34.875772 5937 log.cc:824] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:34.876605 5939 log.cc:824] T e4576eacab0140f1bb0b73bfe801d737 P da343fcf2bf447da8fe7d08885614e50: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:34.876607 5936 log.cc:824] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:34.878675 5937 tablet_bootstrap.cc:492] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147: No bootstrap required, opened a new log
I20260806 16:21:34.878968 5937 ts_tablet_manager.cc:1397] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 16:21:34.879467 5936 tablet_bootstrap.cc:492] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285: No bootstrap required, opened a new log
I20260806 16:21:34.879717 5936 ts_tablet_manager.cc:1397] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260806 16:21:34.882014 5939 tablet_bootstrap.cc:492] T e4576eacab0140f1bb0b73bfe801d737 P da343fcf2bf447da8fe7d08885614e50: No bootstrap required, opened a new log
I20260806 16:21:34.882278 5939 ts_tablet_manager.cc:1397] T e4576eacab0140f1bb0b73bfe801d737 P da343fcf2bf447da8fe7d08885614e50: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260806 16:21:34.883405 5937 raft_consensus.cc:348] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816d0a661a904e849b5b3541e9583285" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37923 } } peers { permanent_uuid: "da343fcf2bf447da8fe7d08885614e50" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33057 } } peers { permanent_uuid: "2c8e598e9e4c42dd9436626e64284147" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41483 } }
I20260806 16:21:34.883755 5937 raft_consensus.cc:374] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:34.883811 5937 raft_consensus.cc:729] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8e598e9e4c42dd9436626e64284147, State: Initialized, Role: FOLLOWER
I20260806 16:21:34.884188 5936 raft_consensus.cc:348] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816d0a661a904e849b5b3541e9583285" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37923 } } peers { permanent_uuid: "da343fcf2bf447da8fe7d08885614e50" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33057 } } peers { permanent_uuid: "2c8e598e9e4c42dd9436626e64284147" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41483 } }
I20260806 16:21:34.884472 5936 raft_consensus.cc:374] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:34.884533 5936 raft_consensus.cc:729] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 816d0a661a904e849b5b3541e9583285, State: Initialized, Role: FOLLOWER
I20260806 16:21:34.884438 5937 consensus_queue.cc:260] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816d0a661a904e849b5b3541e9583285" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37923 } } peers { permanent_uuid: "da343fcf2bf447da8fe7d08885614e50" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33057 } } peers { permanent_uuid: "2c8e598e9e4c42dd9436626e64284147" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41483 } }
I20260806 16:21:34.884990 5936 consensus_queue.cc:260] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816d0a661a904e849b5b3541e9583285" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37923 } } peers { permanent_uuid: "da343fcf2bf447da8fe7d08885614e50" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33057 } } peers { permanent_uuid: "2c8e598e9e4c42dd9436626e64284147" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41483 } }
I20260806 16:21:34.885469 5913 heartbeater.cc:503] Master 127.11.245.190:34061 was elected leader, sending a full tablet report...
I20260806 16:21:34.885870 5937 ts_tablet_manager.cc:1428] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20260806 16:21:34.886106 5936 ts_tablet_manager.cc:1428] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260806 16:21:34.886577 5730 heartbeater.cc:503] Master 127.11.245.190:34061 was elected leader, sending a full tablet report...
I20260806 16:21:34.889439 5939 raft_consensus.cc:348] T e4576eacab0140f1bb0b73bfe801d737 P da343fcf2bf447da8fe7d08885614e50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816d0a661a904e849b5b3541e9583285" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37923 } } peers { permanent_uuid: "da343fcf2bf447da8fe7d08885614e50" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33057 } } peers { permanent_uuid: "2c8e598e9e4c42dd9436626e64284147" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41483 } }
I20260806 16:21:34.889904 5939 raft_consensus.cc:374] T e4576eacab0140f1bb0b73bfe801d737 P da343fcf2bf447da8fe7d08885614e50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:34.889981 5939 raft_consensus.cc:729] T e4576eacab0140f1bb0b73bfe801d737 P da343fcf2bf447da8fe7d08885614e50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da343fcf2bf447da8fe7d08885614e50, State: Initialized, Role: FOLLOWER
I20260806 16:21:34.890550 5939 consensus_queue.cc:260] T e4576eacab0140f1bb0b73bfe801d737 P da343fcf2bf447da8fe7d08885614e50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816d0a661a904e849b5b3541e9583285" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37923 } } peers { permanent_uuid: "da343fcf2bf447da8fe7d08885614e50" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33057 } } peers { permanent_uuid: "2c8e598e9e4c42dd9436626e64284147" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41483 } }
I20260806 16:21:34.891957 5541 heartbeater.cc:503] Master 127.11.245.190:34061 was elected leader, sending a full tablet report...
I20260806 16:21:34.891993 5939 ts_tablet_manager.cc:1428] T e4576eacab0140f1bb0b73bfe801d737 P da343fcf2bf447da8fe7d08885614e50: Time spent starting tablet: real 0.010s user 0.005s sys 0.005s
I20260806 16:21:34.892689 5943 raft_consensus.cc:482] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:21:34.892946 5943 raft_consensus.cc:504] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816d0a661a904e849b5b3541e9583285" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37923 } } peers { permanent_uuid: "da343fcf2bf447da8fe7d08885614e50" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33057 } } peers { permanent_uuid: "2c8e598e9e4c42dd9436626e64284147" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41483 } }
I20260806 16:21:34.894784 5943 leader_election.cc:288] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 816d0a661a904e849b5b3541e9583285 (127.11.245.130:37923), da343fcf2bf447da8fe7d08885614e50 (127.11.245.129:33057)
I20260806 16:21:34.900024 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.895293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:33057 (local address 127.11.245.131:39247)
0806 16:21:34.895647 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:34.895664 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:34.895754 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:34.896482 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:34.896486 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:34.896510 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:34.896655 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:34.896663 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.897727 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.897731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:34.898729 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:34.898740 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.899519 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.899523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:34.899629 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.899657 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 16:21:34.899759 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 16:21:34.899849 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":32}
I20260806 16:21:34.900024 5928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.895888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:39247 (local address 127.11.245.129:33057)
0806 16:21:34.896047 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:34.896052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:34.896118 (+ 66us) server_negotiation.cc:407] Connection header received
0806 16:21:34.896177 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:34.896180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:34.896250 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:34.896332 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:34.896800 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.897590 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.898883 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.899380 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.899531 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.899600 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:34.899778 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 16:21:34.899858 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 16:21:34.901680 5932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.896708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:41409 (local address 127.11.245.130:37923)
0806 16:21:34.896887 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:34.896893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:34.896965 (+ 72us) server_negotiation.cc:407] Connection header received
0806 16:21:34.897203 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:34.897207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:34.897287 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:34.897347 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:34.898014 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.898570 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.900224 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.900891 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.900999 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.901055 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:34.901419 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 16:21:34.901530 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 16:21:34.902179 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.896056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:37923 (local address 127.11.245.131:41409)
0806 16:21:34.896856 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:34.896874 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:34.897049 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:34.897632 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:34.897635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:34.897657 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:34.897869 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:34.897877 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.898717 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.898721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:34.900000 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:34.900012 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.901173 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.901177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:34.901285 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.901307 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 16:21:34.901922 (+ 615us) client_negotiation.cc:240] Negotiation successful
0806 16:21:34.902015 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"mutex_wait_us":22,"thread_start_us":243,"threads_started":1}
I20260806 16:21:34.902590 5471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4576eacab0140f1bb0b73bfe801d737" candidate_uuid: "2c8e598e9e4c42dd9436626e64284147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da343fcf2bf447da8fe7d08885614e50" is_pre_election: true
I20260806 16:21:34.902837 5664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4576eacab0140f1bb0b73bfe801d737" candidate_uuid: "2c8e598e9e4c42dd9436626e64284147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "816d0a661a904e849b5b3541e9583285" is_pre_election: true
I20260806 16:21:34.903095 5471 raft_consensus.cc:2454] T e4576eacab0140f1bb0b73bfe801d737 P da343fcf2bf447da8fe7d08885614e50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8e598e9e4c42dd9436626e64284147 in term 0.
I20260806 16:21:34.903169 5664 raft_consensus.cc:2454] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8e598e9e4c42dd9436626e64284147 in term 0.
I20260806 16:21:34.903777 5760 leader_election.cc:302] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2c8e598e9e4c42dd9436626e64284147, 816d0a661a904e849b5b3541e9583285, da343fcf2bf447da8fe7d08885614e50; no voters:
I20260806 16:21:34.904237 5943 raft_consensus.cc:2781] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:21:34.904336 5943 raft_consensus.cc:482] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:21:34.904376 5943 raft_consensus.cc:3037] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:34.906098 5943 raft_consensus.cc:504] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816d0a661a904e849b5b3541e9583285" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37923 } } peers { permanent_uuid: "da343fcf2bf447da8fe7d08885614e50" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33057 } } peers { permanent_uuid: "2c8e598e9e4c42dd9436626e64284147" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41483 } }
I20260806 16:21:34.906613 5943 leader_election.cc:288] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [CANDIDATE]: Term 1 election: Requested vote from peers 816d0a661a904e849b5b3541e9583285 (127.11.245.130:37923), da343fcf2bf447da8fe7d08885614e50 (127.11.245.129:33057)
I20260806 16:21:34.907202 5664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4576eacab0140f1bb0b73bfe801d737" candidate_uuid: "2c8e598e9e4c42dd9436626e64284147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "816d0a661a904e849b5b3541e9583285"
I20260806 16:21:34.907333 5471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4576eacab0140f1bb0b73bfe801d737" candidate_uuid: "2c8e598e9e4c42dd9436626e64284147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da343fcf2bf447da8fe7d08885614e50"
I20260806 16:21:34.907390 5664 raft_consensus.cc:3037] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:34.907519 5471 raft_consensus.cc:3037] T e4576eacab0140f1bb0b73bfe801d737 P da343fcf2bf447da8fe7d08885614e50 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:34.909015 5664 raft_consensus.cc:2454] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8e598e9e4c42dd9436626e64284147 in term 1.
I20260806 16:21:34.909140 5471 raft_consensus.cc:2454] T e4576eacab0140f1bb0b73bfe801d737 P da343fcf2bf447da8fe7d08885614e50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8e598e9e4c42dd9436626e64284147 in term 1.
I20260806 16:21:34.909566 5760 leader_election.cc:302] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8e598e9e4c42dd9436626e64284147, 816d0a661a904e849b5b3541e9583285; no voters:
I20260806 16:21:34.910023 5943 raft_consensus.cc:2781] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:34.910390 5943 raft_consensus.cc:686] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8e598e9e4c42dd9436626e64284147, State: Running, Role: LEADER
I20260806 16:21:34.910887 5943 consensus_queue.cc:237] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816d0a661a904e849b5b3541e9583285" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37923 } } peers { permanent_uuid: "da343fcf2bf447da8fe7d08885614e50" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33057 } } peers { permanent_uuid: "2c8e598e9e4c42dd9436626e64284147" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41483 } }
I20260806 16:21:34.915894 5271 catalog_manager.cc:5495] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8e598e9e4c42dd9436626e64284147 (127.11.245.131). New cstate: current_term: 1 leader_uuid: "2c8e598e9e4c42dd9436626e64284147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "816d0a661a904e849b5b3541e9583285" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da343fcf2bf447da8fe7d08885614e50" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8e598e9e4c42dd9436626e64284147" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 41483 } health_report { overall_health: HEALTHY } } }
I20260806 16:21:34.938794 5935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:34.934027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58754 (local address 127.11.245.131:41483)
0806 16:21:34.934288 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:34.934293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:34.934309 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:21:34.934422 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:34.934424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:34.934497 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:34.934614 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:21:34.935024 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.935619 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.936824 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:34.937112 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:34.937254 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:34.937607 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:21:34.938290 (+ 683us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:21:34.938418 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:34.938499 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 16:21:34.938571 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260806 16:21:34.947268 5471 raft_consensus.cc:1261] T e4576eacab0140f1bb0b73bfe801d737 P da343fcf2bf447da8fe7d08885614e50 [term 1 FOLLOWER]: Refusing update from remote peer 2c8e598e9e4c42dd9436626e64284147: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:21:34.947268 5664 raft_consensus.cc:1261] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285 [term 1 FOLLOWER]: Refusing update from remote peer 2c8e598e9e4c42dd9436626e64284147: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:21:34.948321 5950 consensus_queue.cc:1035] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da343fcf2bf447da8fe7d08885614e50" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 33057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:21:34.948577 5943 consensus_queue.cc:1035] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [LEADER]: Connected to new peer: Peer: permanent_uuid: "816d0a661a904e849b5b3541e9583285" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:21:34.959213 5955 mvcc.cc:205] Tried to move back new op lower bound from 7315592376093229056 to 7315592375958732800. Current Snapshot: MvccSnapshot[applied={T|T < 7315592376093229056}]
I20260806 16:21:34.971187 5274 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:21:34.971478 5274 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:21:34.974877 5274 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 539a5939ad13443795c0f025f6005628: Sending DeleteTablet for 3 replicas of tablet e4576eacab0140f1bb0b73bfe801d737
I20260806 16:21:34.976114 5637 tablet_service.cc:1514] Processing DeleteTablet for tablet e4576eacab0140f1bb0b73bfe801d737 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:34 UTC) from {username='slave'} at 127.0.0.1:52354
I20260806 16:21:34.976172 5440 tablet_service.cc:1514] Processing DeleteTablet for tablet e4576eacab0140f1bb0b73bfe801d737 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:34 UTC) from {username='slave'} at 127.0.0.1:55012
I20260806 16:21:34.976830 5967 tablet_replica.cc:330] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285: stopping tablet replica
I20260806 16:21:34.976830 5968 tablet_replica.cc:330] T e4576eacab0140f1bb0b73bfe801d737 P da343fcf2bf447da8fe7d08885614e50: stopping tablet replica
I20260806 16:21:34.976850 5816 tablet_service.cc:1514] Processing DeleteTablet for tablet e4576eacab0140f1bb0b73bfe801d737 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:34 UTC) from {username='slave'} at 127.0.0.1:58738
I20260806 16:21:34.977339 5968 raft_consensus.cc:2229] T e4576eacab0140f1bb0b73bfe801d737 P da343fcf2bf447da8fe7d08885614e50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:21:34.977424 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 5351
I20260806 16:21:34.977370 5967 raft_consensus.cc:2229] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:21:34.984742 5967 raft_consensus.cc:2258] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:21:34.988487 5969 tablet_replica.cc:330] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147: stopping tablet replica
I20260806 16:21:34.988994 5967 ts_tablet_manager.cc:1905] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:21:34.989080 5969 raft_consensus.cc:2229] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:21:34.989867 5969 raft_consensus.cc:2258] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:21:34.992188 5969 ts_tablet_manager.cc:1905] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:21:34.993069 5967 ts_tablet_manager.cc:1918] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:21:34.993216 5967 log.cc:1196] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/e4576eacab0140f1bb0b73bfe801d737
I20260806 16:21:34.993667 5967 ts_tablet_manager.cc:1939] T e4576eacab0140f1bb0b73bfe801d737 P 816d0a661a904e849b5b3541e9583285: Deleting consensus metadata
I20260806 16:21:34.995751 5262 catalog_manager.cc:4841] TS 816d0a661a904e849b5b3541e9583285 (127.11.245.130:37923): tablet e4576eacab0140f1bb0b73bfe801d737 (table test-table [id=4ebc30b692b4470e88e48963f4d3bea2]) successfully deleted
I20260806 16:21:34.996197 5969 ts_tablet_manager.cc:1918] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:21:34.996356 5969 log.cc:1196] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/e4576eacab0140f1bb0b73bfe801d737
I20260806 16:21:34.996709 5969 ts_tablet_manager.cc:1939] T e4576eacab0140f1bb0b73bfe801d737 P 2c8e598e9e4c42dd9436626e64284147: Deleting consensus metadata
W20260806 16:21:34.998215 5260 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:33057 (error 108)
I20260806 16:21:34.998541 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 5545
I20260806 16:21:34.998862 5260 catalog_manager.cc:4841] TS 2c8e598e9e4c42dd9436626e64284147 (127.11.245.131:41483): tablet e4576eacab0140f1bb0b73bfe801d737 (table test-table [id=4ebc30b692b4470e88e48963f4d3bea2]) successfully deleted
I20260806 16:21:35.017150 5929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:35.016644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:33057 (local address 127.0.0.1:55026)
0806 16:21:35.016853 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:35.016985 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260806 16:21:35.017462 5260 catalog_manager.cc:4584] TS da343fcf2bf447da8fe7d08885614e50 (127.11.245.129:33057): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4576eacab0140f1bb0b73bfe801d737: Network error: Client connection negotiation failed: client connection to 127.11.245.129:33057: connect: Connection refused (error 111)
I20260806 16:21:35.026734 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 5736
I20260806 16:21:35.045286 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 5234
2026-08-06T16:21:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1435 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:21:35 dist-test-slave-t3q3 krb5kdc[5975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:21:35 dist-test-slave-t3q3 krb5kdc[5975](info): set up 2 sockets
Aug 06 16:21:35 dist-test-slave-t3q3 krb5kdc[5975](info): commencing operation
krb5kdc: starting...
W20260806 16:21:36.341771 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.003s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:21:36 dist-test-slave-t3q3 krb5kdc[5975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:21:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:21:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:21:36.506032 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:43595
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:43595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:36.657092 5994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:36.657529 5994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:36.657613 5994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:36.662302 5994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:21:36.662426 5994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:36.662462 5994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:36.662506 5994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:21:36.662540 5994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:21:36.669353 5994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:43595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:43595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:36.671319 5994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:36.672986 5994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:36.685149 6004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:36.685086 6007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:36.685105 6002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:36.685418 5994 server_base.cc:1034] running on GCE node
I20260806 16:21:36.686463 5994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:36.688041 5994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:36.689328 5994 hybrid_clock.cc:648] HybridClock initialized: now 1786033296689303 us; error 43 us; skew 500 ppm
Aug 06 16:21:36 dist-test-slave-t3q3 krb5kdc[5975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033296, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:36.692896 5994 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:21:36.694200 5994 webserver.cc:460] Webserver started at http://127.11.245.190:45385/ using document root <none> and password file <none>
I20260806 16:21:36.694991 5994 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:36.695093 5994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:36.695391 5994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:36.698028 5994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "0034e811a7654954b4e1e261b004575e"
format_stamp: "Formatted at 2026-08-06 16:21:36 on dist-test-slave-t3q3"
server_key: "846286bc9b6149aa1066ba03b854148b"
server_key_iv: "4149477253fb641b839e1655b3956730"
server_key_version: "encryptionkey@0"
I20260806 16:21:36.698720 5994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "0034e811a7654954b4e1e261b004575e"
format_stamp: "Formatted at 2026-08-06 16:21:36 on dist-test-slave-t3q3"
server_key: "846286bc9b6149aa1066ba03b854148b"
server_key_iv: "4149477253fb641b839e1655b3956730"
server_key_version: "encryptionkey@0"
I20260806 16:21:36.703505 5994 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260806 16:21:36.706804 6014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:36.709021 5994 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:21:36.709213 5994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "0034e811a7654954b4e1e261b004575e"
format_stamp: "Formatted at 2026-08-06 16:21:36 on dist-test-slave-t3q3"
server_key: "846286bc9b6149aa1066ba03b854148b"
server_key_iv: "4149477253fb641b839e1655b3956730"
server_key_version: "encryptionkey@0"
I20260806 16:21:36.709393 5994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:36.725744 5994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:36.730960 5994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:36.731281 5994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:36.743281 5994 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:43595
I20260806 16:21:36.743286 6091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:43595 every 8 connection(s)
I20260806 16:21:36.744658 5994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:21:36.748097 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 5994
I20260806 16:21:36.748313 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:21:36.748768 12246 external_mini_cluster.cc:1428] Setting key ae48ac96b14b63803a4c9029927e3ea1
I20260806 16:21:36.748822 6092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:36.757998 6092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e: Bootstrap starting.
I20260806 16:21:36.762187 6092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e: Neither blocks nor log segments found. Creating new log.
Aug 06 16:21:36 dist-test-slave-t3q3 krb5kdc[5975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:21:36.763461 6092 log.cc:824] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:36.766500 6092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e: No bootstrap required, opened a new log
I20260806 16:21:36.769953 6096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:36.750618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56364 (local address 127.11.245.190:43595)
0806 16:21:36.751208 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:36.751238 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:36.752599 (+ 1361us) server_negotiation.cc:407] Connection header received
0806 16:21:36.753541 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:36.753565 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:36.753982 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:36.754331 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:36.755687 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:36.757869 (+ 2182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:36.759413 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:36.759997 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:36.760314 (+ 317us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:36.764174 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:36.764183 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:36.764206 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:36.764245 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:36.766306 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:36.766941 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:36.766948 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:36.766955 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:36.767093 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:36.767418 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:36.767421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:36.767423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:36.767938 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:36.768205 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:36.768589 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 16:21:36.768966 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":206,"threads_started":1}
I20260806 16:21:36.772241 6092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0034e811a7654954b4e1e261b004575e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43595 } }
I20260806 16:21:36.772639 6092 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:36.772694 6092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0034e811a7654954b4e1e261b004575e, State: Initialized, Role: FOLLOWER
I20260806 16:21:36.773319 6092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0034e811a7654954b4e1e261b004575e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43595 } }
I20260806 16:21:36.773524 6092 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:21:36.773605 6092 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:21:36.773696 6092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:36.774941 6092 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0034e811a7654954b4e1e261b004575e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43595 } }
I20260806 16:21:36.775342 6092 leader_election.cc:302] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0034e811a7654954b4e1e261b004575e; no voters:
I20260806 16:21:36.775760 6092 leader_election.cc:288] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:21:36.775875 6098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:36.776113 6098 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [term 1 LEADER]: Becoming Leader. State: Replica: 0034e811a7654954b4e1e261b004575e, State: Running, Role: LEADER
I20260806 16:21:36.776496 6098 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0034e811a7654954b4e1e261b004575e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43595 } }
I20260806 16:21:36.777546 6092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:21:36.778853 6104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0034e811a7654954b4e1e261b004575e. Latest consensus state: current_term: 1 leader_uuid: "0034e811a7654954b4e1e261b004575e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0034e811a7654954b4e1e261b004575e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43595 } } }
I20260806 16:21:36.779039 6104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:36.779081 6100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0034e811a7654954b4e1e261b004575e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0034e811a7654954b4e1e261b004575e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 43595 } } }
I20260806 16:21:36.779211 6100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:36.779487 6112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:21:36.783636 6112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:21:36.791826 6112 catalog_manager.cc:1347] Generated new cluster ID: 3ce0ecdf9919456786b2001174cdc973
I20260806 16:21:36.791949 6112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:21:36.805357 6112 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:21:36.806803 6112 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:21:36.812090 6112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e: Generated new TSK 0
I20260806 16:21:36.813124 6112 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:21:36.860316 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43595
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:37.034653 6128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:37.035048 6128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:37.035100 6128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:37.039675 6128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:37.039786 6128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:37.039860 6128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:21:37.046555 6128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:37.048154 6128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:37.049556 6128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:37.062718 6140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:37.062745 6137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:37.062731 6136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:37.062940 6128 server_base.cc:1034] running on GCE node
I20260806 16:21:37.063705 6128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:37.064760 6128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:37.066025 6128 hybrid_clock.cc:648] HybridClock initialized: now 1786033297065998 us; error 42 us; skew 500 ppm
Aug 06 16:21:37 dist-test-slave-t3q3 krb5kdc[5975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033297, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:37.070012 6128 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:21:37.071343 6128 webserver.cc:460] Webserver started at http://127.11.245.129:40065/ using document root <none> and password file <none>
I20260806 16:21:37.072268 6128 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:37.072377 6128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:37.072686 6128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:37.075476 6128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "74c18397c95e4de0b4d92fc964376175"
format_stamp: "Formatted at 2026-08-06 16:21:37 on dist-test-slave-t3q3"
server_key: "8d3cd4ebf17b5bc6475d547b76e13948"
server_key_iv: "c3b62f3441f511f41c609257f29f54cd"
server_key_version: "encryptionkey@0"
I20260806 16:21:37.076344 6128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "74c18397c95e4de0b4d92fc964376175"
format_stamp: "Formatted at 2026-08-06 16:21:37 on dist-test-slave-t3q3"
server_key: "8d3cd4ebf17b5bc6475d547b76e13948"
server_key_iv: "c3b62f3441f511f41c609257f29f54cd"
server_key_version: "encryptionkey@0"
I20260806 16:21:37.083242 6128 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 16:21:37.091005 6157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:37.093369 6128 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.000s
I20260806 16:21:37.093590 6128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "74c18397c95e4de0b4d92fc964376175"
format_stamp: "Formatted at 2026-08-06 16:21:37 on dist-test-slave-t3q3"
server_key: "8d3cd4ebf17b5bc6475d547b76e13948"
server_key_iv: "c3b62f3441f511f41c609257f29f54cd"
server_key_version: "encryptionkey@0"
I20260806 16:21:37.093839 6128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:37.114146 6128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:37.118486 6128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:37.118742 6128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:37.119510 6128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:37.120678 6128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:37.120733 6128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:37.120779 6128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:37.120793 6128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:37.132149 6128 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:37973
I20260806 16:21:37.132153 6315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:37973 every 8 connection(s)
I20260806 16:21:37.133437 6128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:21:37.143636 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 6128
I20260806 16:21:37.143829 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:21:37.144304 12246 external_mini_cluster.cc:1428] Setting key a716fec1db5171ec6d777e515ccb1362
Aug 06 16:21:37 dist-test-slave-t3q3 krb5kdc[5975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033297, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:21:37.151464 6096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.135947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:49931 (local address 127.11.245.190:43595)
0806 16:21:37.136181 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:37.136186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:37.137276 (+ 1090us) server_negotiation.cc:407] Connection header received
0806 16:21:37.138855 (+ 1579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:37.138858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:37.138915 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:37.139008 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:37.140527 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.141145 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.142365 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.142630 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.142726 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.146370 (+ 3644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:37.146374 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:37.146376 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:37.146439 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:37.148303 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:37.149150 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:37.149157 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:37.149160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:37.149287 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:37.149777 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:37.149783 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:37.149786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:37.150100 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:37.150212 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:37.150954 (+ 742us) server_negotiation.cc:299] Negotiation successful
0806 16:21:37.151170 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 16:21:37.152801 6319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.136469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43595 (local address 127.11.245.129:49931)
0806 16:21:37.137089 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:37.137141 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:37.138608 (+ 1467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:37.139218 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:37.139225 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:37.139662 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:37.140362 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:37.140381 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.141267 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.141270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:37.142239 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:37.142248 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.142907 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.142910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:37.143102 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.143960 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:21:37.143986 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:37.146121 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:37.148449 (+ 2328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:37.148460 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:37.148491 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:37.148985 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:37.149457 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:37.149460 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:37.149461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:37.149651 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:37.150249 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:37.150257 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:37.150625 (+ 368us) client_negotiation.cc:769] Sending connection context
0806 16:21:37.150882 (+ 257us) client_negotiation.cc:240] Negotiation successful
0806 16:21:37.151271 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":149,"threads_started":1}
I20260806 16:21:37.154831 6316 heartbeater.cc:347] Connected to a master server at 127.11.245.190:43595
I20260806 16:21:37.155203 6316 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:37.155993 6316 heartbeater.cc:511] Master 127.11.245.190:43595 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:21:37.158842 6041 ts_manager.cc:194] Registered new tserver with Master: 74c18397c95e4de0b4d92fc964376175 (127.11.245.129:37973)
I20260806 16:21:37.160879 6041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:49931
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:21:37.184692 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43595
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:37.346463 6322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:37.346824 6322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:37.346879 6322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:37.351303 6322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:37.351398 6322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:37.351467 6322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:21:37.357540 6322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:37.359097 6322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:37.360535 6322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:37.374652 6327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:37.374913 6330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:37.374990 6322 server_base.cc:1034] running on GCE node
W20260806 16:21:37.375000 6328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:37.375857 6322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:37.376879 6322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:37.378206 6322 hybrid_clock.cc:648] HybridClock initialized: now 1786033297378181 us; error 44 us; skew 500 ppm
Aug 06 16:21:37 dist-test-slave-t3q3 krb5kdc[5975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033297, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:37.382486 6322 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:21:37.385197 6322 webserver.cc:460] Webserver started at http://127.11.245.130:36565/ using document root <none> and password file <none>
I20260806 16:21:37.386369 6322 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:37.386490 6322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:37.386785 6322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:37.389801 6322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "7406bc29bf7b40ba9732551171b397eb"
format_stamp: "Formatted at 2026-08-06 16:21:37 on dist-test-slave-t3q3"
server_key: "35642da4d9202ca1315e123ab145e15d"
server_key_iv: "b73b0369669f6fed051d0896e6a384ef"
server_key_version: "encryptionkey@0"
I20260806 16:21:37.390584 6322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "7406bc29bf7b40ba9732551171b397eb"
format_stamp: "Formatted at 2026-08-06 16:21:37 on dist-test-slave-t3q3"
server_key: "35642da4d9202ca1315e123ab145e15d"
server_key_iv: "b73b0369669f6fed051d0896e6a384ef"
server_key_version: "encryptionkey@0"
I20260806 16:21:37.395681 6322 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:21:37.399130 6340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:37.400949 6322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:21:37.401299 6322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "7406bc29bf7b40ba9732551171b397eb"
format_stamp: "Formatted at 2026-08-06 16:21:37 on dist-test-slave-t3q3"
server_key: "35642da4d9202ca1315e123ab145e15d"
server_key_iv: "b73b0369669f6fed051d0896e6a384ef"
server_key_version: "encryptionkey@0"
I20260806 16:21:37.401481 6322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:37.424618 6322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:37.428988 6322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:37.429250 6322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:37.430092 6322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:37.431216 6322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:37.431296 6322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:37.431384 6322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:37.431444 6322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:37.443408 6322 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:37693
I20260806 16:21:37.443465 6529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:37693 every 8 connection(s)
I20260806 16:21:37.444689 6322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:21:37.446645 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 6322
I20260806 16:21:37.446811 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:21:37.447244 12246 external_mini_cluster.cc:1428] Setting key 1f4e078ef30a068b1b7438109b6fcb77
Aug 06 16:21:37 dist-test-slave-t3q3 krb5kdc[5975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033297, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
I20260806 16:21:37.465819 6096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.447333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:49791 (local address 127.11.245.190:43595)
0806 16:21:37.447521 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:37.447527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:37.448782 (+ 1255us) server_negotiation.cc:407] Connection header received
0806 16:21:37.450442 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:37.450448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:37.450524 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:37.450631 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:37.452223 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.453148 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.454588 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.455002 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.455137 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.459038 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:37.459046 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:37.459049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:37.459093 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:37.462117 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:37.463148 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:37.463154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:37.463156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:37.463309 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:37.463820 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:37.463823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:37.463824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:37.464183 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:37.464277 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:37.465252 (+ 975us) server_negotiation.cc:299] Negotiation successful
0806 16:21:37.465529 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:21:37.467022 6533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.447865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43595 (local address 127.11.245.130:49791)
0806 16:21:37.448591 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:37.448656 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:37.450172 (+ 1516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:37.450871 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:37.450901 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:37.451336 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:37.451987 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:37.452005 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.453368 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.453372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:37.454432 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:37.454446 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.455225 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.455232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:37.455456 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.456272 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:21:37.456293 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:37.458762 (+ 2469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:37.462326 (+ 3564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:37.462337 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:37.462370 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:37.462984 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:37.463478 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:37.463482 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:37.463484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:37.463706 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:37.464303 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:37.464310 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:37.464805 (+ 495us) client_negotiation.cc:769] Sending connection context
0806 16:21:37.465155 (+ 350us) client_negotiation.cc:240] Negotiation successful
0806 16:21:37.465680 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":234,"threads_started":1}
I20260806 16:21:37.468750 6530 heartbeater.cc:347] Connected to a master server at 127.11.245.190:43595
I20260806 16:21:37.469142 6530 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:37.469957 6530 heartbeater.cc:511] Master 127.11.245.190:43595 requested a full tablet report, sending...
I20260806 16:21:37.471544 6041 ts_manager.cc:194] Registered new tserver with Master: 7406bc29bf7b40ba9732551171b397eb (127.11.245.130:37693)
I20260806 16:21:37.472386 6041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:49791
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:21:37.487309 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43595
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:37.639355 6538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:37.639719 6538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:37.639770 6538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:37.645540 6538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:37.645666 6538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:37.645740 6538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:21:37.652509 6538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:43595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:37.654146 6538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:37.655522 6538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:37.667778 6546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:37.667778 6544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:37.667806 6543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:37.668323 6538 server_base.cc:1034] running on GCE node
I20260806 16:21:37.668869 6538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:37.669760 6538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:37.670979 6538 hybrid_clock.cc:648] HybridClock initialized: now 1786033297670956 us; error 35 us; skew 500 ppm
Aug 06 16:21:37 dist-test-slave-t3q3 krb5kdc[5975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033297, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:37.674467 6538 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:21:37.675717 6538 webserver.cc:460] Webserver started at http://127.11.245.131:40607/ using document root <none> and password file <none>
I20260806 16:21:37.676539 6538 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:37.676613 6538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:37.676841 6538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:37.680365 6538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "9f5efd5f112240b7bc54260e584378cf"
format_stamp: "Formatted at 2026-08-06 16:21:37 on dist-test-slave-t3q3"
server_key: "92dbc087106affe7e263210eb3ab0a92"
server_key_iv: "d1e6730ac7c0e4debd1c0dd93526edef"
server_key_version: "encryptionkey@0"
I20260806 16:21:37.681088 6538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "9f5efd5f112240b7bc54260e584378cf"
format_stamp: "Formatted at 2026-08-06 16:21:37 on dist-test-slave-t3q3"
server_key: "92dbc087106affe7e263210eb3ab0a92"
server_key_iv: "d1e6730ac7c0e4debd1c0dd93526edef"
server_key_version: "encryptionkey@0"
I20260806 16:21:37.687496 6538 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 16:21:37.691288 6556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:37.693157 6538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:21:37.693356 6538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "9f5efd5f112240b7bc54260e584378cf"
format_stamp: "Formatted at 2026-08-06 16:21:37 on dist-test-slave-t3q3"
server_key: "92dbc087106affe7e263210eb3ab0a92"
server_key_iv: "d1e6730ac7c0e4debd1c0dd93526edef"
server_key_version: "encryptionkey@0"
I20260806 16:21:37.693501 6538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:37.731154 6538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:37.736311 6538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:37.736610 6538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:37.737509 6538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:37.739029 6538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:37.739090 6538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:37.739348 6538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:37.739401 6538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:37.751883 6538 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:40117
I20260806 16:21:37.751940 6717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:40117 every 8 connection(s)
I20260806 16:21:37.753299 6538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:21:37.759565 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 6538
I20260806 16:21:37.759770 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:21:37.760221 12246 external_mini_cluster.cc:1428] Setting key b8f1eaad3a40d5cdc8490b24998120b8
Aug 06 16:21:37 dist-test-slave-t3q3 krb5kdc[5975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033297, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:21:37.771061 6096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.755833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:44237 (local address 127.11.245.190:43595)
0806 16:21:37.756008 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:37.756012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:37.757300 (+ 1288us) server_negotiation.cc:407] Connection header received
0806 16:21:37.758919 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:37.758923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:37.758993 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:37.759110 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:37.760801 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.761502 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.762843 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.763148 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.763242 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.766317 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:37.766320 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:37.766323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:37.766358 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:37.768021 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:37.768811 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:37.768814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:37.768816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:37.768908 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:37.769373 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:37.769377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:37.769385 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:37.769635 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:37.769723 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:37.770599 (+ 876us) server_negotiation.cc:299] Negotiation successful
0806 16:21:37.770779 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 16:21:37.772440 6722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.756468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:43595 (local address 127.11.245.131:44237)
0806 16:21:37.757117 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:37.757167 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:37.758669 (+ 1502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:37.759346 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:37.759353 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:37.759893 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:37.760626 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:37.760645 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.761654 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.761658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:37.762699 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:37.762715 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.763308 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.763312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:37.763481 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.764271 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:21:37.764290 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:37.766114 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:37.768252 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:37.768260 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:37.768279 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:37.768648 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:37.769033 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:37.769037 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:37.769039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:37.769226 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:37.769790 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:37.769797 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:37.770238 (+ 441us) client_negotiation.cc:769] Sending connection context
0806 16:21:37.770497 (+ 259us) client_negotiation.cc:240] Negotiation successful
0806 16:21:37.770907 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":177,"threads_started":1}
I20260806 16:21:37.774272 6718 heartbeater.cc:347] Connected to a master server at 127.11.245.190:43595
I20260806 16:21:37.774693 6718 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:37.775571 6718 heartbeater.cc:511] Master 127.11.245.190:43595 requested a full tablet report, sending...
I20260806 16:21:37.777004 6041 ts_manager.cc:194] Registered new tserver with Master: 9f5efd5f112240b7bc54260e584378cf (127.11.245.131:40117)
I20260806 16:21:37.777884 6041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:44237
I20260806 16:21:37.788426 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:21:37.801246 6096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.791148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56370 (local address 127.11.245.190:43595)
0806 16:21:37.791308 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:37.791312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:37.791460 (+ 148us) server_negotiation.cc:407] Connection header received
0806 16:21:37.791574 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:37.791577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:37.791635 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:37.791714 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:37.792694 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.793298 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.794250 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.794518 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.794627 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.796756 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:37.796760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:37.796762 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:37.796793 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:37.798895 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:37.799653 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:37.799658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:37.799660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:37.799762 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:37.800150 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:37.800153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:37.800155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:37.800432 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:37.800618 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:37.800916 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 16:21:37.801046 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 16:21:37.805895 6041 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:21:37.810127 6041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:21:37.831610 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.825185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47406 (local address 127.11.245.130:37693)
0806 16:21:37.825924 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:37.825944 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:37.825964 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:21:37.826048 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:37.826052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:37.826252 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:37.826429 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:37.826647 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.827976 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.829859 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.830734 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.830885 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.831078 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:37.831264 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 16:21:37.831400 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":189,"threads_started":1}
I20260806 16:21:37.831857 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.824257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44666 (local address 127.11.245.129:37973)
0806 16:21:37.824866 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:37.824878 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:37.825395 (+ 517us) server_negotiation.cc:407] Connection header received
0806 16:21:37.825796 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:37.825807 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:37.826103 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:37.826250 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:37.826695 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.827720 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.829702 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.830734 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.830885 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.831079 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:37.831544 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 16:21:37.831682 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":192,"threads_started":1}
I20260806 16:21:37.832015 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.825195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33014 (local address 127.11.245.131:40117)
0806 16:21:37.825857 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:37.825872 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:37.826166 (+ 294us) server_negotiation.cc:407] Connection header received
0806 16:21:37.826288 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:37.826294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:37.826488 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:37.826647 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:37.826938 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.828629 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.830167 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.830827 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.830969 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.831119 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:37.831686 (+ 567us) server_negotiation.cc:299] Negotiation successful
0806 16:21:37.831818 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":174,"threads_started":1}
I20260806 16:21:37.831602 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.823786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:37693 (local address 127.0.0.1:47406)
0806 16:21:37.825743 (+ 1957us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:37.825758 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:37.825847 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:37.826444 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:37.826447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:37.826460 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:37.826563 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:37.826568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.828273 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.828277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:37.829710 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:37.829723 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.830899 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.830906 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:37.831020 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.831063 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 16:21:37.831209 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 16:21:37.831392 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1887,"mutex_wait_us":36,"thread_start_us":530,"threads_started":1}
I20260806 16:21:37.832315 6737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.824498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:40117 (local address 127.0.0.1:33014)
0806 16:21:37.826047 (+ 1549us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:37.826059 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:37.826167 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:37.826684 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:37.826687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:37.826703 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:37.826804 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:37.826810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.828811 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.828815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:37.830035 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:37.830044 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.831189 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.831194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:37.831389 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.831411 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 16:21:37.831488 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 16:21:37.831555 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1485,"mutex_wait_us":59,"thread_start_us":140,"threads_started":1}
I20260806 16:21:37.832640 6738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.823745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:37973 (local address 127.0.0.1:44666)
0806 16:21:37.825199 (+ 1454us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:37.825265 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:37.825560 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:37.826333 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:37.826339 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:37.826361 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:37.826554 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:37.826562 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.827843 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.827848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:37.829548 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:37.829560 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.831195 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.831202 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:37.831347 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.831382 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 16:21:37.832007 (+ 625us) client_negotiation.cc:240] Negotiation successful
0806 16:21:37.832130 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1225,"mutex_wait_us":65,"thread_start_us":399,"threads_started":1}
I20260806 16:21:37.836408 6224 tablet_service.cc:1467] Processing CreateTablet for tablet 61631813760f449d88838de4fdc9b065 (DEFAULT_TABLE table=test-table [id=7deb38fb9f4149368dcca7ee74428dfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:37.836410 6627 tablet_service.cc:1467] Processing CreateTablet for tablet 61631813760f449d88838de4fdc9b065 (DEFAULT_TABLE table=test-table [id=7deb38fb9f4149368dcca7ee74428dfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:37.836409 6413 tablet_service.cc:1467] Processing CreateTablet for tablet 61631813760f449d88838de4fdc9b065 (DEFAULT_TABLE table=test-table [id=7deb38fb9f4149368dcca7ee74428dfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:37.838012 6413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61631813760f449d88838de4fdc9b065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:37.838008 6627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61631813760f449d88838de4fdc9b065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:37.838007 6224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61631813760f449d88838de4fdc9b065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:37.847354 6744 tablet_bootstrap.cc:492] T 61631813760f449d88838de4fdc9b065 P 74c18397c95e4de0b4d92fc964376175: Bootstrap starting.
I20260806 16:21:37.848215 6746 tablet_bootstrap.cc:492] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf: Bootstrap starting.
I20260806 16:21:37.848937 6745 tablet_bootstrap.cc:492] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb: Bootstrap starting.
I20260806 16:21:37.851296 6745 tablet_bootstrap.cc:654] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:37.851297 6746 tablet_bootstrap.cc:654] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:37.852838 6745 log.cc:824] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:37.852838 6746 log.cc:824] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:37.855945 6746 tablet_bootstrap.cc:492] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf: No bootstrap required, opened a new log
I20260806 16:21:37.855952 6745 tablet_bootstrap.cc:492] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb: No bootstrap required, opened a new log
I20260806 16:21:37.856271 6745 ts_tablet_manager.cc:1397] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 16:21:37.856681 6744 tablet_bootstrap.cc:654] T 61631813760f449d88838de4fdc9b065 P 74c18397c95e4de0b4d92fc964376175: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:37.856257 6746 ts_tablet_manager.cc:1397] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.003s
I20260806 16:21:37.858019 6744 log.cc:824] T 61631813760f449d88838de4fdc9b065 P 74c18397c95e4de0b4d92fc964376175: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:37.861092 6745 raft_consensus.cc:348] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c18397c95e4de0b4d92fc964376175" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37973 } } peers { permanent_uuid: "7406bc29bf7b40ba9732551171b397eb" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37693 } } peers { permanent_uuid: "9f5efd5f112240b7bc54260e584378cf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40117 } }
I20260806 16:21:37.861450 6745 raft_consensus.cc:374] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:37.861513 6745 raft_consensus.cc:729] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7406bc29bf7b40ba9732551171b397eb, State: Initialized, Role: FOLLOWER
I20260806 16:21:37.861616 6744 tablet_bootstrap.cc:492] T 61631813760f449d88838de4fdc9b065 P 74c18397c95e4de0b4d92fc964376175: No bootstrap required, opened a new log
I20260806 16:21:37.861919 6744 ts_tablet_manager.cc:1397] T 61631813760f449d88838de4fdc9b065 P 74c18397c95e4de0b4d92fc964376175: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s
I20260806 16:21:37.862196 6745 consensus_queue.cc:260] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c18397c95e4de0b4d92fc964376175" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37973 } } peers { permanent_uuid: "7406bc29bf7b40ba9732551171b397eb" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37693 } } peers { permanent_uuid: "9f5efd5f112240b7bc54260e584378cf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40117 } }
I20260806 16:21:37.862620 6746 raft_consensus.cc:348] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c18397c95e4de0b4d92fc964376175" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37973 } } peers { permanent_uuid: "7406bc29bf7b40ba9732551171b397eb" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37693 } } peers { permanent_uuid: "9f5efd5f112240b7bc54260e584378cf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40117 } }
I20260806 16:21:37.863179 6530 heartbeater.cc:503] Master 127.11.245.190:43595 was elected leader, sending a full tablet report...
I20260806 16:21:37.864204 6746 raft_consensus.cc:374] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:37.864387 6745 ts_tablet_manager.cc:1428] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 16:21:37.864396 6746 raft_consensus.cc:729] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f5efd5f112240b7bc54260e584378cf, State: Initialized, Role: FOLLOWER
I20260806 16:21:37.865145 6746 consensus_queue.cc:260] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c18397c95e4de0b4d92fc964376175" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37973 } } peers { permanent_uuid: "7406bc29bf7b40ba9732551171b397eb" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37693 } } peers { permanent_uuid: "9f5efd5f112240b7bc54260e584378cf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40117 } }
I20260806 16:21:37.866748 6746 ts_tablet_manager.cc:1428] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
I20260806 16:21:37.867250 6718 heartbeater.cc:503] Master 127.11.245.190:43595 was elected leader, sending a full tablet report...
I20260806 16:21:37.867856 6744 raft_consensus.cc:348] T 61631813760f449d88838de4fdc9b065 P 74c18397c95e4de0b4d92fc964376175 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c18397c95e4de0b4d92fc964376175" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37973 } } peers { permanent_uuid: "7406bc29bf7b40ba9732551171b397eb" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37693 } } peers { permanent_uuid: "9f5efd5f112240b7bc54260e584378cf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40117 } }
I20260806 16:21:37.868342 6744 raft_consensus.cc:374] T 61631813760f449d88838de4fdc9b065 P 74c18397c95e4de0b4d92fc964376175 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:37.868418 6744 raft_consensus.cc:729] T 61631813760f449d88838de4fdc9b065 P 74c18397c95e4de0b4d92fc964376175 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74c18397c95e4de0b4d92fc964376175, State: Initialized, Role: FOLLOWER
I20260806 16:21:37.869017 6744 consensus_queue.cc:260] T 61631813760f449d88838de4fdc9b065 P 74c18397c95e4de0b4d92fc964376175 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c18397c95e4de0b4d92fc964376175" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37973 } } peers { permanent_uuid: "7406bc29bf7b40ba9732551171b397eb" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37693 } } peers { permanent_uuid: "9f5efd5f112240b7bc54260e584378cf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40117 } }
I20260806 16:21:37.870431 6744 ts_tablet_manager.cc:1428] T 61631813760f449d88838de4fdc9b065 P 74c18397c95e4de0b4d92fc964376175: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260806 16:21:37.870491 6316 heartbeater.cc:503] Master 127.11.245.190:43595 was elected leader, sending a full tablet report...
W20260806 16:21:37.886305 6318 tablet.cc:2369] T 61631813760f449d88838de4fdc9b065 P 74c18397c95e4de0b4d92fc964376175: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:21:37.946455 6532 tablet.cc:2369] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:21:37.958046 6751 raft_consensus.cc:482] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:21:37.958360 6751 raft_consensus.cc:504] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c18397c95e4de0b4d92fc964376175" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37973 } } peers { permanent_uuid: "7406bc29bf7b40ba9732551171b397eb" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37693 } } peers { permanent_uuid: "9f5efd5f112240b7bc54260e584378cf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40117 } }
I20260806 16:21:37.960196 6751 leader_election.cc:288] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74c18397c95e4de0b4d92fc964376175 (127.11.245.129:37973), 9f5efd5f112240b7bc54260e584378cf (127.11.245.131:40117)
I20260806 16:21:37.964881 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.960746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:59947 (local address 127.11.245.131:40117)
0806 16:21:37.960918 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:37.960924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:37.960945 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:21:37.961040 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:37.961043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:37.961106 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:37.961359 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:37.961719 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.962301 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.963704 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.964197 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.964350 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.964420 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:37.964648 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 16:21:37.964746 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 16:21:37.964881 6533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.960612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:40117 (local address 127.11.245.130:59947)
0806 16:21:37.960770 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:37.960790 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:37.960927 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:37.961358 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:37.961362 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:37.961384 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:37.961578 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:37.961585 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.962434 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.962446 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:37.963530 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:37.963547 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.964347 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.964352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:37.964507 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.964524 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 16:21:37.964625 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 16:21:37.964711 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":154}
I20260806 16:21:37.964881 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.960947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:37711 (local address 127.11.245.129:37973)
0806 16:21:37.961097 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:37.961102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:37.961313 (+ 211us) server_negotiation.cc:407] Connection header received
0806 16:21:37.961476 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:37.961481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:37.961538 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:37.961632 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:37.961956 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.962483 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.963653 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.964182 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.964328 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.964404 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:37.964617 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 16:21:37.964712 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 16:21:37.964892 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.960626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:37973 (local address 127.11.245.130:37711)
0806 16:21:37.961189 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:37.961200 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:37.961342 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:37.961647 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:37.961650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:37.961667 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:37.961826 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:37.961832 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.962612 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.962617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:37.963506 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:37.963517 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.964328 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.964332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:37.964466 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.964489 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 16:21:37.964595 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 16:21:37.964687 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":151,"thread_start_us":104,"threads_started":1}
I20260806 16:21:37.965859 6656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61631813760f449d88838de4fdc9b065" candidate_uuid: "7406bc29bf7b40ba9732551171b397eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f5efd5f112240b7bc54260e584378cf" is_pre_election: true
I20260806 16:21:37.965860 6252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61631813760f449d88838de4fdc9b065" candidate_uuid: "7406bc29bf7b40ba9732551171b397eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74c18397c95e4de0b4d92fc964376175" is_pre_election: true
I20260806 16:21:37.966322 6656 raft_consensus.cc:2454] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7406bc29bf7b40ba9732551171b397eb in term 0.
I20260806 16:21:37.966321 6252 raft_consensus.cc:2454] T 61631813760f449d88838de4fdc9b065 P 74c18397c95e4de0b4d92fc964376175 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7406bc29bf7b40ba9732551171b397eb in term 0.
I20260806 16:21:37.967018 6341 leader_election.cc:302] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7406bc29bf7b40ba9732551171b397eb, 74c18397c95e4de0b4d92fc964376175, 9f5efd5f112240b7bc54260e584378cf; no voters:
I20260806 16:21:37.967407 6751 raft_consensus.cc:2781] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:21:37.967521 6751 raft_consensus.cc:482] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:21:37.967571 6751 raft_consensus.cc:3037] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:37.969060 6751 raft_consensus.cc:504] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c18397c95e4de0b4d92fc964376175" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37973 } } peers { permanent_uuid: "7406bc29bf7b40ba9732551171b397eb" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37693 } } peers { permanent_uuid: "9f5efd5f112240b7bc54260e584378cf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40117 } }
I20260806 16:21:37.969543 6751 leader_election.cc:288] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [CANDIDATE]: Term 1 election: Requested vote from peers 74c18397c95e4de0b4d92fc964376175 (127.11.245.129:37973), 9f5efd5f112240b7bc54260e584378cf (127.11.245.131:40117)
I20260806 16:21:37.970072 6252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61631813760f449d88838de4fdc9b065" candidate_uuid: "7406bc29bf7b40ba9732551171b397eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74c18397c95e4de0b4d92fc964376175"
I20260806 16:21:37.970134 6656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61631813760f449d88838de4fdc9b065" candidate_uuid: "7406bc29bf7b40ba9732551171b397eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f5efd5f112240b7bc54260e584378cf"
I20260806 16:21:37.970314 6656 raft_consensus.cc:3037] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:37.970381 6252 raft_consensus.cc:3037] T 61631813760f449d88838de4fdc9b065 P 74c18397c95e4de0b4d92fc964376175 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:37.971772 6656 raft_consensus.cc:2454] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7406bc29bf7b40ba9732551171b397eb in term 1.
I20260806 16:21:37.971961 6252 raft_consensus.cc:2454] T 61631813760f449d88838de4fdc9b065 P 74c18397c95e4de0b4d92fc964376175 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7406bc29bf7b40ba9732551171b397eb in term 1.
I20260806 16:21:37.972326 6344 leader_election.cc:302] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7406bc29bf7b40ba9732551171b397eb, 9f5efd5f112240b7bc54260e584378cf; no voters:
I20260806 16:21:37.972618 6751 raft_consensus.cc:2781] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:37.972973 6751 raft_consensus.cc:686] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [term 1 LEADER]: Becoming Leader. State: Replica: 7406bc29bf7b40ba9732551171b397eb, State: Running, Role: LEADER
I20260806 16:21:37.973472 6751 consensus_queue.cc:237] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c18397c95e4de0b4d92fc964376175" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37973 } } peers { permanent_uuid: "7406bc29bf7b40ba9732551171b397eb" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37693 } } peers { permanent_uuid: "9f5efd5f112240b7bc54260e584378cf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40117 } }
I20260806 16:21:37.978056 6038 catalog_manager.cc:5495] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 7406bc29bf7b40ba9732551171b397eb (127.11.245.130). New cstate: current_term: 1 leader_uuid: "7406bc29bf7b40ba9732551171b397eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c18397c95e4de0b4d92fc964376175" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7406bc29bf7b40ba9732551171b397eb" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f5efd5f112240b7bc54260e584378cf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40117 } health_report { overall_health: UNKNOWN } } }
I20260806 16:21:37.996613 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:37.991669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47410 (local address 127.11.245.130:37693)
0806 16:21:37.991927 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:37.991932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:37.991948 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:21:37.992134 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:37.992137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:37.992222 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:37.992342 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:21:37.992718 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.993311 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.994250 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:37.994509 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:37.994611 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:37.994983 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:21:37.995587 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:21:37.995727 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:37.996305 (+ 578us) server_negotiation.cc:299] Negotiation successful
0806 16:21:37.996383 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
W20260806 16:21:38.004590 6721 tablet.cc:2369] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:21:38.005905 6656 raft_consensus.cc:1261] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf [term 1 FOLLOWER]: Refusing update from remote peer 7406bc29bf7b40ba9732551171b397eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:21:38.005908 6252 raft_consensus.cc:1261] T 61631813760f449d88838de4fdc9b065 P 74c18397c95e4de0b4d92fc964376175 [term 1 FOLLOWER]: Refusing update from remote peer 7406bc29bf7b40ba9732551171b397eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:21:38.006964 6751 consensus_queue.cc:1035] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f5efd5f112240b7bc54260e584378cf" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:21:38.007275 6755 consensus_queue.cc:1035] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c18397c95e4de0b4d92fc964376175" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 37973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:21:38.020531 6762 mvcc.cc:205] Tried to move back new op lower bound from 7315592388620320768 to 7315592388503015424. Current Snapshot: MvccSnapshot[applied={T|T < 7315592388620320768}]
I20260806 16:21:38.029332 6040 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:21:38.029583 6040 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:21:38.033061 6040 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0034e811a7654954b4e1e261b004575e: Sending DeleteTablet for 3 replicas of tablet 61631813760f449d88838de4fdc9b065
I20260806 16:21:38.034265 6627 tablet_service.cc:1514] Processing DeleteTablet for tablet 61631813760f449d88838de4fdc9b065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:38 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:33014
I20260806 16:21:38.034265 6413 tablet_service.cc:1514] Processing DeleteTablet for tablet 61631813760f449d88838de4fdc9b065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:38 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:47406
I20260806 16:21:38.034264 6224 tablet_service.cc:1514] Processing DeleteTablet for tablet 61631813760f449d88838de4fdc9b065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:38 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:44666
I20260806 16:21:38.035043 6779 tablet_replica.cc:330] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf: stopping tablet replica
I20260806 16:21:38.035223 6778 tablet_replica.cc:330] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb: stopping tablet replica
I20260806 16:21:38.035547 6779 raft_consensus.cc:2229] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:21:38.035547 6778 raft_consensus.cc:2229] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:21:38.035828 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 6128
I20260806 16:21:38.039656 6779 raft_consensus.cc:2258] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:21:38.046860 6778 raft_consensus.cc:2258] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:21:38.048141 6779 ts_tablet_manager.cc:1905] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:21:38.051257 6778 ts_tablet_manager.cc:1905] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:21:38.052412 6779 ts_tablet_manager.cc:1918] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:21:38.052640 6779 log.cc:1196] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/61631813760f449d88838de4fdc9b065
I20260806 16:21:38.053119 6779 ts_tablet_manager.cc:1939] T 61631813760f449d88838de4fdc9b065 P 9f5efd5f112240b7bc54260e584378cf: Deleting consensus metadata
I20260806 16:21:38.054711 6778 ts_tablet_manager.cc:1918] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:21:38.054834 6778 log.cc:1196] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/61631813760f449d88838de4fdc9b065
I20260806 16:21:38.055241 6778 ts_tablet_manager.cc:1939] T 61631813760f449d88838de4fdc9b065 P 7406bc29bf7b40ba9732551171b397eb: Deleting consensus metadata
I20260806 16:21:38.055292 6016 catalog_manager.cc:4841] TS 9f5efd5f112240b7bc54260e584378cf (127.11.245.131:40117): tablet 61631813760f449d88838de4fdc9b065 (table test-table [id=7deb38fb9f4149368dcca7ee74428dfa]) successfully deleted
I20260806 16:21:38.056869 6018 catalog_manager.cc:4841] TS 7406bc29bf7b40ba9732551171b397eb (127.11.245.130:37693): tablet 61631813760f449d88838de4fdc9b065 (table test-table [id=7deb38fb9f4149368dcca7ee74428dfa]) successfully deleted
W20260806 16:21:38.068038 6015 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:37973 (error 108)
I20260806 16:21:38.068186 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 6322
I20260806 16:21:38.076749 6737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:38.076315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:37973 (local address 127.0.0.1:44678)
0806 16:21:38.076518 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:38.076622 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 16:21:38.077023 6015 catalog_manager.cc:4584] TS 74c18397c95e4de0b4d92fc964376175 (127.11.245.129:37973): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 61631813760f449d88838de4fdc9b065: Network error: Client connection negotiation failed: client connection to 127.11.245.129:37973: connect: Connection refused (error 111)
I20260806 16:21:38.086035 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 6538
I20260806 16:21:38.103576 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 5994
2026-08-06T16:21:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (3060 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T16:21:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:21:38Z Disabled control of system clock
I20260806 16:21:38.157191 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:41999
--webserver_interface=127.11.245.190
--webserver_port=0
--builtin_ntp_servers=127.11.245.148:37709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:41999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:21:38.303395 6785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:38.303797 6785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:38.303839 6785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:38.308208 6785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:21:38.308300 6785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:38.308318 6785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:38.308337 6785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:21:38.308353 6785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:21:38.313668 6785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:41999
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:41999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:38.315276 6785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:38.317147 6785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:38.327874 6793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:38.327912 6794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:38.327868 6797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:38.328140 6785 server_base.cc:1034] running on GCE node
I20260806 16:21:38.329119 6785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:38.330626 6785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:38.331840 6785 hybrid_clock.cc:648] HybridClock initialized: now 1786033298331801 us; error 54 us; skew 500 ppm
I20260806 16:21:38.334640 6785 webserver.cc:460] Webserver started at http://127.11.245.190:37739/ using document root <none> and password file <none>
I20260806 16:21:38.335616 6785 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:38.335762 6785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:38.336162 6785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:38.339159 6785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "608ba6a4879d4cf685bda67502b7c10e"
format_stamp: "Formatted at 2026-08-06 16:21:38 on dist-test-slave-t3q3"
server_key: "6e6c01bcf3085b6eea6ea5ea175675d7"
server_key_iv: "cff04dd3ce2dea6b2d6d05305287a024"
server_key_version: "encryptionkey@0"
I20260806 16:21:38.339949 6785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "608ba6a4879d4cf685bda67502b7c10e"
format_stamp: "Formatted at 2026-08-06 16:21:38 on dist-test-slave-t3q3"
server_key: "6e6c01bcf3085b6eea6ea5ea175675d7"
server_key_iv: "cff04dd3ce2dea6b2d6d05305287a024"
server_key_version: "encryptionkey@0"
I20260806 16:21:38.345180 6785 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 16:21:38.348438 6805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:38.349977 6785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 16:21:38.350145 6785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "608ba6a4879d4cf685bda67502b7c10e"
format_stamp: "Formatted at 2026-08-06 16:21:38 on dist-test-slave-t3q3"
server_key: "6e6c01bcf3085b6eea6ea5ea175675d7"
server_key_iv: "cff04dd3ce2dea6b2d6d05305287a024"
server_key_version: "encryptionkey@0"
I20260806 16:21:38.350288 6785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:38.371713 6785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:38.372737 6785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:38.372991 6785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:38.384708 6785 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:41999
I20260806 16:21:38.384768 6877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:41999 every 8 connection(s)
I20260806 16:21:38.386144 6785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:21:38.387760 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 6785
I20260806 16:21:38.387974 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:21:38.388419 12246 external_mini_cluster.cc:1428] Setting key 44462b96d9227144c0448fc03d7c5ffd
I20260806 16:21:38.391227 6878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:38.399520 6878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e: Bootstrap starting.
I20260806 16:21:38.400394 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:38.390313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49662 (local address 127.11.245.190:41999)
0806 16:21:38.391175 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:38.391195 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:38.391242 (+ 47us) server_negotiation.cc:407] Connection header received
0806 16:21:38.392226 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:38.392250 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:38.392753 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:38.393194 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:38.393727 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:38.394810 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:38.395926 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:38.396667 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:38.396891 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:38.398024 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:38.398030 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:38.398049 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:38.398573 (+ 524us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:38.398646 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:38.398670 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:38.398799 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:38.398981 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 16:21:38.399321 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":196,"threads_started":1}
I20260806 16:21:38.402918 6878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:38.404037 6878 log.cc:824] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:38.407007 6878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e: No bootstrap required, opened a new log
I20260806 16:21:38.411413 6878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608ba6a4879d4cf685bda67502b7c10e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41999 } }
I20260806 16:21:38.411764 6878 raft_consensus.cc:374] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:38.411873 6878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 608ba6a4879d4cf685bda67502b7c10e, State: Initialized, Role: FOLLOWER
I20260806 16:21:38.412539 6878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608ba6a4879d4cf685bda67502b7c10e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41999 } }
I20260806 16:21:38.412730 6878 raft_consensus.cc:388] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:21:38.412827 6878 raft_consensus.cc:482] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:21:38.412983 6878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:38.414507 6878 raft_consensus.cc:504] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608ba6a4879d4cf685bda67502b7c10e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41999 } }
I20260806 16:21:38.414966 6878 leader_election.cc:302] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 608ba6a4879d4cf685bda67502b7c10e; no voters:
I20260806 16:21:38.415392 6878 leader_election.cc:288] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:21:38.415608 6884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:38.415865 6884 raft_consensus.cc:686] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [term 1 LEADER]: Becoming Leader. State: Replica: 608ba6a4879d4cf685bda67502b7c10e, State: Running, Role: LEADER
I20260806 16:21:38.416317 6884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608ba6a4879d4cf685bda67502b7c10e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41999 } }
I20260806 16:21:38.417102 6878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:21:38.418460 6891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 608ba6a4879d4cf685bda67502b7c10e. Latest consensus state: current_term: 1 leader_uuid: "608ba6a4879d4cf685bda67502b7c10e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608ba6a4879d4cf685bda67502b7c10e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41999 } } }
I20260806 16:21:38.418615 6891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:38.418948 6885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "608ba6a4879d4cf685bda67502b7c10e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608ba6a4879d4cf685bda67502b7c10e" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41999 } } }
I20260806 16:21:38.419076 6901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:21:38.419114 6885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:38.423012 6901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:21:38.431324 6901 catalog_manager.cc:1347] Generated new cluster ID: e0ee9b9f91f143609b7824f53893ffe6
I20260806 16:21:38.431490 6901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:21:38.468158 6901 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:21:38.469463 6901 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:21:38.478304 6901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e: Generated new TSK 0
I20260806 16:21:38.479409 6901 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:21:38.494765 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:41999
--builtin_ntp_servers=127.11.245.148:37709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:21:38.641057 6917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:38.641470 6917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:38.641556 6917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:38.645839 6917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:38.645953 6917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:38.646061 6917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:21:38.652215 6917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:41999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:38.654120 6917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:38.656128 6917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:38.668460 6925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:38.668466 6926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:38.668560 6917 server_base.cc:1034] running on GCE node
W20260806 16:21:38.668458 6930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:38.669291 6917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:38.670153 6917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:38.671443 6917 hybrid_clock.cc:648] HybridClock initialized: now 1786033298671423 us; error 38 us; skew 500 ppm
I20260806 16:21:38.673947 6917 webserver.cc:460] Webserver started at http://127.11.245.129:35233/ using document root <none> and password file <none>
I20260806 16:21:38.674737 6917 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:38.674846 6917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:38.675150 6917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:38.678298 6917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "3fa8208046b04c1c9795d857772a0c6b"
format_stamp: "Formatted at 2026-08-06 16:21:38 on dist-test-slave-t3q3"
server_key: "c732dc90d2ba7030f9a16f4069e64659"
server_key_iv: "07d621ddacf4aa56a560cf3f7c9bdf97"
server_key_version: "encryptionkey@0"
I20260806 16:21:38.679080 6917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "3fa8208046b04c1c9795d857772a0c6b"
format_stamp: "Formatted at 2026-08-06 16:21:38 on dist-test-slave-t3q3"
server_key: "c732dc90d2ba7030f9a16f4069e64659"
server_key_iv: "07d621ddacf4aa56a560cf3f7c9bdf97"
server_key_version: "encryptionkey@0"
I20260806 16:21:38.684134 6917 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:21:38.687431 6937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:38.689178 6917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:21:38.689385 6917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "3fa8208046b04c1c9795d857772a0c6b"
format_stamp: "Formatted at 2026-08-06 16:21:38 on dist-test-slave-t3q3"
server_key: "c732dc90d2ba7030f9a16f4069e64659"
server_key_iv: "07d621ddacf4aa56a560cf3f7c9bdf97"
server_key_version: "encryptionkey@0"
I20260806 16:21:38.689548 6917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:38.712896 6917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:38.713945 6917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:38.714210 6917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:38.715037 6917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:38.716459 6917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:38.716542 6917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:38.716645 6917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:38.716691 6917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:38.728763 6917 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:42425
I20260806 16:21:38.728825 7094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:42425 every 8 connection(s)
I20260806 16:21:38.730610 6917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:21:38.736889 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 6917
I20260806 16:21:38.737082 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:21:38.737542 12246 external_mini_cluster.cc:1428] Setting key ed18f6baf8905a1ad38b456a43cc6c73
I20260806 16:21:38.741202 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:41999
--builtin_ntp_servers=127.11.245.148:37709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:21:38.749254 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:38.733689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:49243 (local address 127.11.245.190:41999)
0806 16:21:38.733921 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:38.733928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:38.736424 (+ 2496us) server_negotiation.cc:407] Connection header received
0806 16:21:38.740463 (+ 4039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:38.740468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:38.740544 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:38.740618 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:38.742758 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:38.743729 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:38.746235 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:38.746718 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:38.746870 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:38.748539 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:38.748543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:38.748546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:38.748574 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:38.748600 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:38.748610 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:38.748695 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:38.748914 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 16:21:38.749051 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 16:21:38.750770 7097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:38.734243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:41999 (local address 127.11.245.129:49243)
0806 16:21:38.736137 (+ 1894us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:38.736218 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:38.740093 (+ 3875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:38.741338 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:38.741355 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:38.741521 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:38.742536 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:38.742566 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:38.743962 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:38.743967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:38.746044 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:38.746062 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:38.747002 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:38.747010 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:38.747224 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:38.748132 (+ 908us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:21:38.748134 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:38.748214 (+ 80us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:21:38.748325 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:38.748714 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:38.748726 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:38.748734 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 16:21:38.748907 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 16:21:38.749682 (+ 775us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1149,"thread_start_us":157,"threads_started":1}
I20260806 16:21:38.752504 7095 heartbeater.cc:347] Connected to a master server at 127.11.245.190:41999
I20260806 16:21:38.752828 7095 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:38.753603 7095 heartbeater.cc:511] Master 127.11.245.190:41999 requested a full tablet report, sending...
I20260806 16:21:38.756050 6830 ts_manager.cc:194] Registered new tserver with Master: 3fa8208046b04c1c9795d857772a0c6b (127.11.245.129:42425)
I20260806 16:21:38.757846 6830 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.245.129:49243
W20260806 16:21:38.914403 7098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:38.914844 7098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:38.914903 7098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:38.919862 7098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:38.919955 7098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:38.920028 7098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:21:38.925729 7098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:41999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:38.927323 7098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:38.929304 7098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:38.942457 7104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:38.942481 7108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:38.943112 7110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:38.943229 7098 server_base.cc:1034] running on GCE node
I20260806 16:21:38.943871 7098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:38.944851 7098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:38.946051 7098 hybrid_clock.cc:648] HybridClock initialized: now 1786033298946028 us; error 39 us; skew 500 ppm
I20260806 16:21:38.951493 7098 webserver.cc:460] Webserver started at http://127.11.245.130:36309/ using document root <none> and password file <none>
I20260806 16:21:38.952478 7098 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:38.952566 7098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:38.952852 7098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:38.955971 7098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "e4d5d3949e0d44e4b036a256f30793b3"
format_stamp: "Formatted at 2026-08-06 16:21:38 on dist-test-slave-t3q3"
server_key: "c69f8e9598c55672d2a02a7a920f6888"
server_key_iv: "e0e554c0060ee40222c33e51d45784d2"
server_key_version: "encryptionkey@0"
I20260806 16:21:38.956816 7098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "e4d5d3949e0d44e4b036a256f30793b3"
format_stamp: "Formatted at 2026-08-06 16:21:38 on dist-test-slave-t3q3"
server_key: "c69f8e9598c55672d2a02a7a920f6888"
server_key_iv: "e0e554c0060ee40222c33e51d45784d2"
server_key_version: "encryptionkey@0"
I20260806 16:21:38.962543 7098 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 16:21:38.966284 7119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:38.968127 7098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:21:38.968315 7098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "e4d5d3949e0d44e4b036a256f30793b3"
format_stamp: "Formatted at 2026-08-06 16:21:38 on dist-test-slave-t3q3"
server_key: "c69f8e9598c55672d2a02a7a920f6888"
server_key_iv: "e0e554c0060ee40222c33e51d45784d2"
server_key_version: "encryptionkey@0"
I20260806 16:21:38.968489 7098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:38.994287 7098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:38.995452 7098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:38.995748 7098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:38.996737 7098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:38.998144 7098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:38.998234 7098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:38.998324 7098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:38.998381 7098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:39.011262 7098 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:43751
I20260806 16:21:39.011291 7283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:43751 every 8 connection(s)
I20260806 16:21:39.012634 7098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:21:39.019567 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 7098
I20260806 16:21:39.019734 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:21:39.020437 12246 external_mini_cluster.cc:1428] Setting key ecb5a4bfb2ef7c58f88a0050b82542a2
I20260806 16:21:39.023523 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:41999
--builtin_ntp_servers=127.11.245.148:37709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:21:39.025920 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.015325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:35107 (local address 127.11.245.190:41999)
0806 16:21:39.015557 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:39.015564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:39.017004 (+ 1440us) server_negotiation.cc:407] Connection header received
0806 16:21:39.018889 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:39.018892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:39.018965 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:39.019058 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:39.020759 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.021480 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.022986 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.023392 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.023547 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:39.025178 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:39.025186 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:39.025193 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:39.025215 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:39.025247 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:39.025262 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:39.025362 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:39.025551 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 16:21:39.025651 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 16:21:39.027002 7288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.015902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:41999 (local address 127.11.245.130:35107)
0806 16:21:39.016791 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:39.016868 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:39.018614 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:39.019482 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:39.019492 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:39.019623 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:39.020454 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:39.020472 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.021632 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.021637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:39.022772 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:39.022784 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.023640 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.023646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:39.023917 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:39.024802 (+ 885us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:21:39.024811 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:39.024870 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:21:39.024967 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:39.025365 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:39.025373 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:39.025379 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 16:21:39.025500 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 16:21:39.025835 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":166,"threads_started":1}
I20260806 16:21:39.028590 7285 heartbeater.cc:347] Connected to a master server at 127.11.245.190:41999
I20260806 16:21:39.028920 7285 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:39.029645 7285 heartbeater.cc:511] Master 127.11.245.190:41999 requested a full tablet report, sending...
I20260806 16:21:39.031183 6830 ts_manager.cc:194] Registered new tserver with Master: e4d5d3949e0d44e4b036a256f30793b3 (127.11.245.130:43751)
I20260806 16:21:39.032244 6830 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.245.130:35107
W20260806 16:21:39.209209 7289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:39.209623 7289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:39.209672 7289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:39.214479 7289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:39.214597 7289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:39.214668 7289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:21:39.220919 7289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:37709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:41999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:39.222584 7289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:39.224535 7289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:39.238432 7301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:39.238432 7295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:39.238633 7289 server_base.cc:1034] running on GCE node
W20260806 16:21:39.238859 7297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:39.239432 7289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:39.240429 7289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:39.241681 7289 hybrid_clock.cc:648] HybridClock initialized: now 1786033299241663 us; error 32 us; skew 500 ppm
I20260806 16:21:39.249292 7289 webserver.cc:460] Webserver started at http://127.11.245.131:42473/ using document root <none> and password file <none>
I20260806 16:21:39.250222 7289 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:39.250350 7289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:39.250691 7289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:39.253958 7289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "2db42262fbc845ed80d76cd8427b82ad"
format_stamp: "Formatted at 2026-08-06 16:21:39 on dist-test-slave-t3q3"
server_key: "24cec52c11e17e088a361e2735eb796d"
server_key_iv: "e0d5042be44f41a8912c287f095a501e"
server_key_version: "encryptionkey@0"
I20260806 16:21:39.254827 7289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "2db42262fbc845ed80d76cd8427b82ad"
format_stamp: "Formatted at 2026-08-06 16:21:39 on dist-test-slave-t3q3"
server_key: "24cec52c11e17e088a361e2735eb796d"
server_key_iv: "e0d5042be44f41a8912c287f095a501e"
server_key_version: "encryptionkey@0"
I20260806 16:21:39.260741 7289 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260806 16:21:39.264786 7311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:39.266624 7289 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:21:39.266804 7289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "2db42262fbc845ed80d76cd8427b82ad"
format_stamp: "Formatted at 2026-08-06 16:21:39 on dist-test-slave-t3q3"
server_key: "24cec52c11e17e088a361e2735eb796d"
server_key_iv: "e0d5042be44f41a8912c287f095a501e"
server_key_version: "encryptionkey@0"
I20260806 16:21:39.267000 7289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:39.291574 7289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:39.292775 7289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:39.293076 7289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:39.294024 7289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:39.295774 7289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:39.295832 7289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:39.295919 7289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:39.295953 7289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:39.309473 7289 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:42085
I20260806 16:21:39.309514 7478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:42085 every 8 connection(s)
I20260806 16:21:39.311091 7289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:21:39.319828 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 7289
I20260806 16:21:39.319968 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:21:39.320607 12246 external_mini_cluster.cc:1428] Setting key 0ee4ef063bcb5422a01c340d1fc15347
I20260806 16:21:39.324199 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.313821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:60021 (local address 127.11.245.190:41999)
0806 16:21:39.314008 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:39.314013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:39.315547 (+ 1534us) server_negotiation.cc:407] Connection header received
0806 16:21:39.317492 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:39.317497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:39.317579 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:39.317693 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:39.319284 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.319925 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.321566 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.321805 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.321956 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:39.323554 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:39.323558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:39.323561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:39.323575 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:39.323599 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:39.323607 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:39.323690 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:39.323899 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 16:21:39.323986 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 16:21:39.325423 7482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.314273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:41999 (local address 127.11.245.131:60021)
0806 16:21:39.315315 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:39.315393 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:39.317190 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:39.318146 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:39.318157 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:39.318281 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:39.319063 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:39.319085 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.320150 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.320154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:39.321427 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:39.321440 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.322022 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.322027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:39.322256 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:39.323116 (+ 860us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:21:39.323118 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:39.323202 (+ 84us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:21:39.323328 (+ 126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:39.323710 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:39.323722 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:39.323730 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 16:21:39.323899 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 16:21:39.324343 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":199,"threads_started":1}
I20260806 16:21:39.326854 7479 heartbeater.cc:347] Connected to a master server at 127.11.245.190:41999
I20260806 16:21:39.327203 7479 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:39.327947 7479 heartbeater.cc:511] Master 127.11.245.190:41999 requested a full tablet report, sending...
I20260806 16:21:39.329766 6830 ts_manager.cc:194] Registered new tserver with Master: 2db42262fbc845ed80d76cd8427b82ad (127.11.245.131:42085)
I20260806 16:21:39.330464 6830 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.245.131:60021
I20260806 16:21:39.335973 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:21:39.344756 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.338985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49672 (local address 127.11.245.190:41999)
0806 16:21:39.339167 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:39.339174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:39.339252 (+ 78us) server_negotiation.cc:407] Connection header received
0806 16:21:39.339474 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:39.339477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:39.339536 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:39.339615 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:39.340178 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.340933 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.341900 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.342531 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.342742 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:39.343807 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:39.343810 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:39.343813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:39.343829 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:39.343851 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:39.343871 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:39.343993 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:39.344438 (+ 445us) server_negotiation.cc:299] Negotiation successful
0806 16:21:39.344530 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 16:21:39.349658 6830 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:21:39.354034 6830 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:21:39.373142 7498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.366325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:42425 (local address 127.0.0.1:37472)
0806 16:21:39.367056 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:39.367070 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:39.367155 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:39.367658 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:39.367667 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:39.367697 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:39.367903 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:39.367911 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.369567 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.369569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:39.370675 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:39.370686 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.372151 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.372156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:39.372418 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:39.372663 (+ 245us) client_negotiation.cc:769] Sending connection context
0806 16:21:39.372747 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 16:21:39.372926 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":351,"thread_start_us":96,"threads_started":1}
I20260806 16:21:39.373163 7505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.366727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37472 (local address 127.11.245.129:42425)
0806 16:21:39.367180 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:39.367186 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:39.367210 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:21:39.367291 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:39.367296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:39.367468 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:39.367615 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:39.368045 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.369404 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.370835 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.371820 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.371989 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:39.372603 (+ 614us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:39.372816 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 16:21:39.372962 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":119,"threads_started":1}
I20260806 16:21:39.373347 7499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.365725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42576 (local address 127.11.245.131:42085)
0806 16:21:39.366460 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:39.366476 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:39.366720 (+ 244us) server_negotiation.cc:407] Connection header received
0806 16:21:39.367069 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:39.367075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:39.367344 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:39.367494 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:39.368127 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.369413 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.371263 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.372129 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.372254 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:39.372603 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:39.372934 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 16:21:39.373074 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":207,"threads_started":1}
I20260806 16:21:39.373706 7507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.366717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54958 (local address 127.11.245.130:43751)
0806 16:21:39.367277 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:39.367295 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:39.367318 (+ 23us) server_negotiation.cc:407] Connection header received
0806 16:21:39.367399 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:39.367402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:39.367552 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:39.367742 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:39.368150 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.369299 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.371307 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.372132 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.372873 (+ 741us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:39.373086 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:39.373141 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 16:21:39.373532 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":183,"threads_started":1}
I20260806 16:21:39.373965 7500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.365456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:42085 (local address 127.0.0.1:42576)
0806 16:21:39.366542 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:39.366604 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:39.366836 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:39.367726 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:39.367732 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:39.367755 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:39.367900 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:39.367910 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.369539 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.369542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:39.371111 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:39.371126 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.372674 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.372677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:39.372777 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:39.372796 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 16:21:39.373578 (+ 782us) client_negotiation.cc:240] Negotiation successful
0806 16:21:39.373671 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"mutex_wait_us":110,"spinlock_wait_cycles":768,"thread_start_us":115,"threads_started":1}
I20260806 16:21:39.375056 7503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.365882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:43751 (local address 127.0.0.1:54958)
0806 16:21:39.366657 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:39.366672 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:39.366844 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:39.367779 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:39.367781 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:39.367794 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:39.367987 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:39.367993 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.369420 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.369427 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:39.371162 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:39.371172 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.372416 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.372420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:39.372534 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:39.372599 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 16:21:39.372700 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 16:21:39.374917 (+ 2217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"mutex_wait_us":2317,"thread_start_us":118,"threads_started":1}
I20260806 16:21:39.377820 7382 tablet_service.cc:1467] Processing CreateTablet for tablet d3e2d95d958f4d3eafca14aa1b27cc3e (DEFAULT_TABLE table=test-table [id=03c48d879fb14c15a0afc474b00e8259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:39.377825 7001 tablet_service.cc:1467] Processing CreateTablet for tablet d3e2d95d958f4d3eafca14aa1b27cc3e (DEFAULT_TABLE table=test-table [id=03c48d879fb14c15a0afc474b00e8259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:39.377873 7185 tablet_service.cc:1467] Processing CreateTablet for tablet d3e2d95d958f4d3eafca14aa1b27cc3e (DEFAULT_TABLE table=test-table [id=03c48d879fb14c15a0afc474b00e8259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:39.379304 7382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e2d95d958f4d3eafca14aa1b27cc3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:39.379304 7001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e2d95d958f4d3eafca14aa1b27cc3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:39.379304 7185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e2d95d958f4d3eafca14aa1b27cc3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:39.390542 7509 tablet_bootstrap.cc:492] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3: Bootstrap starting.
I20260806 16:21:39.392570 7508 tablet_bootstrap.cc:492] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b: Bootstrap starting.
I20260806 16:21:39.393785 7509 tablet_bootstrap.cc:654] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:39.395088 7509 log.cc:824] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:39.395745 7510 tablet_bootstrap.cc:492] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad: Bootstrap starting.
I20260806 16:21:39.396047 7508 tablet_bootstrap.cc:654] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:39.397079 7508 log.cc:824] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:39.398033 7509 tablet_bootstrap.cc:492] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3: No bootstrap required, opened a new log
I20260806 16:21:39.398288 7509 ts_tablet_manager.cc:1397] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 16:21:39.399451 7508 tablet_bootstrap.cc:492] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b: No bootstrap required, opened a new log
I20260806 16:21:39.399673 7508 ts_tablet_manager.cc:1397] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 16:21:39.400520 7510 tablet_bootstrap.cc:654] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:39.401535 7510 log.cc:824] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:39.403811 7510 tablet_bootstrap.cc:492] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad: No bootstrap required, opened a new log
I20260806 16:21:39.403684 7509 raft_consensus.cc:348] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db42262fbc845ed80d76cd8427b82ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 42085 } } peers { permanent_uuid: "3fa8208046b04c1c9795d857772a0c6b" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 42425 } } peers { permanent_uuid: "e4d5d3949e0d44e4b036a256f30793b3" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 43751 } }
I20260806 16:21:39.404047 7509 raft_consensus.cc:374] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:39.404129 7510 ts_tablet_manager.cc:1397] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 16:21:39.404140 7509 raft_consensus.cc:729] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4d5d3949e0d44e4b036a256f30793b3, State: Initialized, Role: FOLLOWER
I20260806 16:21:39.404805 7509 consensus_queue.cc:260] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db42262fbc845ed80d76cd8427b82ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 42085 } } peers { permanent_uuid: "3fa8208046b04c1c9795d857772a0c6b" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 42425 } } peers { permanent_uuid: "e4d5d3949e0d44e4b036a256f30793b3" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 43751 } }
I20260806 16:21:39.404712 7508 raft_consensus.cc:348] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db42262fbc845ed80d76cd8427b82ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 42085 } } peers { permanent_uuid: "3fa8208046b04c1c9795d857772a0c6b" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 42425 } } peers { permanent_uuid: "e4d5d3949e0d44e4b036a256f30793b3" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 43751 } }
I20260806 16:21:39.404997 7508 raft_consensus.cc:374] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:39.405071 7508 raft_consensus.cc:729] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fa8208046b04c1c9795d857772a0c6b, State: Initialized, Role: FOLLOWER
I20260806 16:21:39.405596 7508 consensus_queue.cc:260] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db42262fbc845ed80d76cd8427b82ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 42085 } } peers { permanent_uuid: "3fa8208046b04c1c9795d857772a0c6b" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 42425 } } peers { permanent_uuid: "e4d5d3949e0d44e4b036a256f30793b3" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 43751 } }
I20260806 16:21:39.406137 7509 ts_tablet_manager.cc:1428] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3: Time spent starting tablet: real 0.008s user 0.005s sys 0.005s
I20260806 16:21:39.406629 7508 ts_tablet_manager.cc:1428] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 16:21:39.406733 7095 heartbeater.cc:503] Master 127.11.245.190:41999 was elected leader, sending a full tablet report...
I20260806 16:21:39.407648 7285 heartbeater.cc:503] Master 127.11.245.190:41999 was elected leader, sending a full tablet report...
I20260806 16:21:39.408766 7510 raft_consensus.cc:348] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db42262fbc845ed80d76cd8427b82ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 42085 } } peers { permanent_uuid: "3fa8208046b04c1c9795d857772a0c6b" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 42425 } } peers { permanent_uuid: "e4d5d3949e0d44e4b036a256f30793b3" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 43751 } }
I20260806 16:21:39.409052 7510 raft_consensus.cc:374] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:39.409102 7510 raft_consensus.cc:729] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2db42262fbc845ed80d76cd8427b82ad, State: Initialized, Role: FOLLOWER
I20260806 16:21:39.409586 7510 consensus_queue.cc:260] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db42262fbc845ed80d76cd8427b82ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 42085 } } peers { permanent_uuid: "3fa8208046b04c1c9795d857772a0c6b" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 42425 } } peers { permanent_uuid: "e4d5d3949e0d44e4b036a256f30793b3" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 43751 } }
I20260806 16:21:39.410480 7479 heartbeater.cc:503] Master 127.11.245.190:41999 was elected leader, sending a full tablet report...
I20260806 16:21:39.410775 7510 ts_tablet_manager.cc:1428] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 16:21:39.412099 7520 raft_consensus.cc:482] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:21:39.412388 7520 raft_consensus.cc:504] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db42262fbc845ed80d76cd8427b82ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 42085 } } peers { permanent_uuid: "3fa8208046b04c1c9795d857772a0c6b" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 42425 } } peers { permanent_uuid: "e4d5d3949e0d44e4b036a256f30793b3" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 43751 } }
I20260806 16:21:39.414076 7520 leader_election.cc:288] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2db42262fbc845ed80d76cd8427b82ad (127.11.245.131:42085), e4d5d3949e0d44e4b036a256f30793b3 (127.11.245.130:43751)
I20260806 16:21:39.419312 7499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.414777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:54265 (local address 127.11.245.131:42085)
0806 16:21:39.414917 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:39.414922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:39.414939 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:21:39.415116 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:39.415119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:39.415179 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:39.415263 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:39.415771 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.416408 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.418085 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.418646 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.418783 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:39.418854 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:39.419086 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 16:21:39.419181 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 16:21:39.419346 7524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.414303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:42085 (local address 127.11.245.129:54265)
0806 16:21:39.414685 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:39.414705 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:39.414979 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:39.415417 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:39.415423 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:39.415446 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:39.415638 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:39.415651 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.416526 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.416530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:39.417958 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:39.417971 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.418783 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.418787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:39.418892 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:39.418921 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 16:21:39.419033 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 16:21:39.419133 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":50,"thread_start_us":168,"threads_started":1}
I20260806 16:21:39.419406 7525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.414357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:43751 (local address 127.11.245.129:43703)
0806 16:21:39.415055 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:39.415070 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:39.415174 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:39.415599 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:39.415601 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:39.415620 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:39.415732 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:39.415737 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.416582 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.416586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:39.417956 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:39.417969 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.418944 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.418946 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:39.419025 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:39.419040 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 16:21:39.419164 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 16:21:39.419244 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":105,"thread_start_us":124,"threads_started":1}
I20260806 16:21:39.419457 7507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.414797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:43703 (local address 127.11.245.130:43751)
0806 16:21:39.414929 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:39.414934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:39.415140 (+ 206us) server_negotiation.cc:407] Connection header received
0806 16:21:39.415281 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:39.415284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:39.415338 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:39.415447 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:39.415860 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.416465 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.418138 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.418726 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.418814 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:39.418857 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:39.419191 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 16:21:39.419308 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 16:21:39.420333 7216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e2d95d958f4d3eafca14aa1b27cc3e" candidate_uuid: "3fa8208046b04c1c9795d857772a0c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4d5d3949e0d44e4b036a256f30793b3" is_pre_election: true
I20260806 16:21:39.420326 7412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e2d95d958f4d3eafca14aa1b27cc3e" candidate_uuid: "3fa8208046b04c1c9795d857772a0c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2db42262fbc845ed80d76cd8427b82ad" is_pre_election: true
I20260806 16:21:39.420765 7412 raft_consensus.cc:2454] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fa8208046b04c1c9795d857772a0c6b in term 0.
I20260806 16:21:39.420765 7216 raft_consensus.cc:2454] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fa8208046b04c1c9795d857772a0c6b in term 0.
I20260806 16:21:39.421586 6942 leader_election.cc:302] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2db42262fbc845ed80d76cd8427b82ad, 3fa8208046b04c1c9795d857772a0c6b, e4d5d3949e0d44e4b036a256f30793b3; no voters:
I20260806 16:21:39.422012 7520 raft_consensus.cc:2781] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:21:39.422133 7520 raft_consensus.cc:482] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:21:39.422188 7520 raft_consensus.cc:3037] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:39.423633 7520 raft_consensus.cc:504] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db42262fbc845ed80d76cd8427b82ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 42085 } } peers { permanent_uuid: "3fa8208046b04c1c9795d857772a0c6b" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 42425 } } peers { permanent_uuid: "e4d5d3949e0d44e4b036a256f30793b3" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 43751 } }
I20260806 16:21:39.424153 7520 leader_election.cc:288] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [CANDIDATE]: Term 1 election: Requested vote from peers 2db42262fbc845ed80d76cd8427b82ad (127.11.245.131:42085), e4d5d3949e0d44e4b036a256f30793b3 (127.11.245.130:43751)
I20260806 16:21:39.424597 7412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e2d95d958f4d3eafca14aa1b27cc3e" candidate_uuid: "3fa8208046b04c1c9795d857772a0c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2db42262fbc845ed80d76cd8427b82ad"
I20260806 16:21:39.424603 7216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e2d95d958f4d3eafca14aa1b27cc3e" candidate_uuid: "3fa8208046b04c1c9795d857772a0c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4d5d3949e0d44e4b036a256f30793b3"
I20260806 16:21:39.424777 7216 raft_consensus.cc:3037] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:39.424769 7412 raft_consensus.cc:3037] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:39.426107 7216 raft_consensus.cc:2454] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fa8208046b04c1c9795d857772a0c6b in term 1.
I20260806 16:21:39.426113 7412 raft_consensus.cc:2454] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fa8208046b04c1c9795d857772a0c6b in term 1.
I20260806 16:21:39.426520 6941 leader_election.cc:302] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fa8208046b04c1c9795d857772a0c6b, e4d5d3949e0d44e4b036a256f30793b3; no voters:
I20260806 16:21:39.426739 7520 raft_consensus.cc:2781] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:39.427042 7520 raft_consensus.cc:686] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [term 1 LEADER]: Becoming Leader. State: Replica: 3fa8208046b04c1c9795d857772a0c6b, State: Running, Role: LEADER
I20260806 16:21:39.427479 7520 consensus_queue.cc:237] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db42262fbc845ed80d76cd8427b82ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 42085 } } peers { permanent_uuid: "3fa8208046b04c1c9795d857772a0c6b" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 42425 } } peers { permanent_uuid: "e4d5d3949e0d44e4b036a256f30793b3" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 43751 } }
I20260806 16:21:39.431861 6830 catalog_manager.cc:5495] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fa8208046b04c1c9795d857772a0c6b (127.11.245.129). New cstate: current_term: 1 leader_uuid: "3fa8208046b04c1c9795d857772a0c6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db42262fbc845ed80d76cd8427b82ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 42085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fa8208046b04c1c9795d857772a0c6b" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4d5d3949e0d44e4b036a256f30793b3" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 43751 } health_report { overall_health: UNKNOWN } } }
I20260806 16:21:39.459036 7505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.454596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37482 (local address 127.11.245.129:42425)
0806 16:21:39.454760 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:39.454764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:39.454779 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:21:39.454918 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:39.454920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:39.454999 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:39.455115 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:21:39.455496 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.456156 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.457211 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:39.457495 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:39.457593 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:39.457928 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:21:39.458518 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:21:39.458647 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:39.458821 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 16:21:39.458870 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 16:21:39.467700 7216 raft_consensus.cc:1261] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3 [term 1 FOLLOWER]: Refusing update from remote peer 3fa8208046b04c1c9795d857772a0c6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:21:39.467700 7412 raft_consensus.cc:1261] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad [term 1 FOLLOWER]: Refusing update from remote peer 3fa8208046b04c1c9795d857772a0c6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:21:39.468777 7526 consensus_queue.cc:1035] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4d5d3949e0d44e4b036a256f30793b3" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 43751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:21:39.469069 7520 consensus_queue.cc:1035] T d3e2d95d958f4d3eafca14aa1b27cc3e P 3fa8208046b04c1c9795d857772a0c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2db42262fbc845ed80d76cd8427b82ad" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 42085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:21:39.497051 6829 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:21:39.497355 6829 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:21:39.501024 6829 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 608ba6a4879d4cf685bda67502b7c10e: Sending DeleteTablet for 3 replicas of tablet d3e2d95d958f4d3eafca14aa1b27cc3e
I20260806 16:21:39.502314 7185 tablet_service.cc:1514] Processing DeleteTablet for tablet d3e2d95d958f4d3eafca14aa1b27cc3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:39 UTC) from {username='slave'} at 127.0.0.1:54958
I20260806 16:21:39.502355 7382 tablet_service.cc:1514] Processing DeleteTablet for tablet d3e2d95d958f4d3eafca14aa1b27cc3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:39 UTC) from {username='slave'} at 127.0.0.1:42576
I20260806 16:21:39.503126 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 6917
I20260806 16:21:39.503123 7541 tablet_replica.cc:330] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad: stopping tablet replica
I20260806 16:21:39.514679 7543 tablet_replica.cc:330] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3: stopping tablet replica
I20260806 16:21:39.514714 7541 raft_consensus.cc:2229] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:21:39.515519 7541 raft_consensus.cc:2258] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:21:39.515514 7543 raft_consensus.cc:2229] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:21:39.515941 7543 raft_consensus.cc:2258] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:21:39.517904 7541 ts_tablet_manager.cc:1905] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:21:39.517920 7543 ts_tablet_manager.cc:1905] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:21:39.521132 7543 ts_tablet_manager.cc:1918] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:21:39.521283 7543 log.cc:1196] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/d3e2d95d958f4d3eafca14aa1b27cc3e
I20260806 16:21:39.521623 7543 ts_tablet_manager.cc:1939] T d3e2d95d958f4d3eafca14aa1b27cc3e P e4d5d3949e0d44e4b036a256f30793b3: Deleting consensus metadata
I20260806 16:21:39.522199 7541 ts_tablet_manager.cc:1918] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:21:39.522321 7541 log.cc:1196] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/d3e2d95d958f4d3eafca14aa1b27cc3e
I20260806 16:21:39.522683 7541 ts_tablet_manager.cc:1939] T d3e2d95d958f4d3eafca14aa1b27cc3e P 2db42262fbc845ed80d76cd8427b82ad: Deleting consensus metadata
I20260806 16:21:39.523432 6810 catalog_manager.cc:4841] TS e4d5d3949e0d44e4b036a256f30793b3 (127.11.245.130:43751): tablet d3e2d95d958f4d3eafca14aa1b27cc3e (table test-table [id=03c48d879fb14c15a0afc474b00e8259]) successfully deleted
I20260806 16:21:39.524314 6814 catalog_manager.cc:4841] TS 2db42262fbc845ed80d76cd8427b82ad (127.11.245.131:42085): tablet d3e2d95d958f4d3eafca14aa1b27cc3e (table test-table [id=03c48d879fb14c15a0afc474b00e8259]) successfully deleted
W20260806 16:21:39.527925 6814 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:42425 (error 108)
I20260806 16:21:39.527987 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 7098
I20260806 16:21:39.535435 7498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:39.533871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:42425 (local address 127.0.0.1:37492)
0806 16:21:39.535076 (+ 1205us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:39.535193 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:42425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1027}
W20260806 16:21:39.541301 6814 catalog_manager.cc:4584] TS 3fa8208046b04c1c9795d857772a0c6b (127.11.245.129:42425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d3e2d95d958f4d3eafca14aa1b27cc3e: Network error: Client connection negotiation failed: client connection to 127.11.245.129:42425: connect: Connection refused (error 111)
I20260806 16:21:39.551631 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 7289
I20260806 16:21:39.569794 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 6785
2026-08-06T16:21:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1462 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:21:39 dist-test-slave-t3q3 krb5kdc[7547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:21:39 dist-test-slave-t3q3 krb5kdc[7547](info): set up 2 sockets
Aug 06 16:21:39 dist-test-slave-t3q3 krb5kdc[7547](info): commencing operation
krb5kdc: starting...
W20260806 16:21:40.891031 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:21:41 dist-test-slave-t3q3 krb5kdc[7547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:21:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:21:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:21:41.058061 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:45971
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:45971
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:41.211349 7566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:41.211827 7566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:41.211918 7566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:41.216809 7566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:21:41.216980 7566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:41.217027 7566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:41.217070 7566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:21:41.217110 7566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:21:41.224375 7566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:45971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:45971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:41.226019 7566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:41.227419 7566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:41.239018 7572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:41.239006 7576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:41.239006 7573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:41.239498 7566 server_base.cc:1034] running on GCE node
I20260806 16:21:41.240363 7566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:41.241997 7566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:41.243496 7566 hybrid_clock.cc:648] HybridClock initialized: now 1786033301243438 us; error 64 us; skew 500 ppm
Aug 06 16:21:41 dist-test-slave-t3q3 krb5kdc[7547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033301, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:41.247509 7566 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:21:41.248862 7566 webserver.cc:460] Webserver started at http://127.11.245.190:45241/ using document root <none> and password file <none>
I20260806 16:21:41.249655 7566 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:41.249768 7566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:41.250077 7566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:41.252977 7566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "6b0325220a3445bf938655ffe31d0eec"
format_stamp: "Formatted at 2026-08-06 16:21:41 on dist-test-slave-t3q3"
server_key: "622954c902724cfa59fd328fe4b78707"
server_key_iv: "5725d85950f58db0706885ab650ec91a"
server_key_version: "encryptionkey@0"
I20260806 16:21:41.253798 7566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "6b0325220a3445bf938655ffe31d0eec"
format_stamp: "Formatted at 2026-08-06 16:21:41 on dist-test-slave-t3q3"
server_key: "622954c902724cfa59fd328fe4b78707"
server_key_iv: "5725d85950f58db0706885ab650ec91a"
server_key_version: "encryptionkey@0"
I20260806 16:21:41.259871 7566 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 16:21:41.264401 7589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:41.266402 7566 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:21:41.266652 7566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "6b0325220a3445bf938655ffe31d0eec"
format_stamp: "Formatted at 2026-08-06 16:21:41 on dist-test-slave-t3q3"
server_key: "622954c902724cfa59fd328fe4b78707"
server_key_iv: "5725d85950f58db0706885ab650ec91a"
server_key_version: "encryptionkey@0"
I20260806 16:21:41.266841 7566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:41.288837 7566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:41.293344 7566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:41.293635 7566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:41.305976 7566 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:45971
I20260806 16:21:41.305994 7673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:45971 every 8 connection(s)
I20260806 16:21:41.307379 7566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:21:41.310556 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 7566
I20260806 16:21:41.310740 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:21:41.311120 12246 external_mini_cluster.cc:1428] Setting key 48037ee3285866d073d718a5ce9dad2d
I20260806 16:21:41.311664 7674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:41.319976 7674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec: Bootstrap starting.
Aug 06 16:21:41 dist-test-slave-t3q3 krb5kdc[7547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:21:41.323493 7674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:41.324854 7674 log.cc:824] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:41.327584 7674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec: No bootstrap required, opened a new log
I20260806 16:21:41.329190 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:41.313112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51248 (local address 127.11.245.190:45971)
0806 16:21:41.313724 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:41.313744 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:41.313790 (+ 46us) server_negotiation.cc:407] Connection header received
0806 16:21:41.314630 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:41.314653 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:41.315078 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:41.315526 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:41.316520 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:41.317622 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:41.318563 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:41.319102 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:41.319726 (+ 624us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:41.322819 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:41.322825 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:41.322845 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:41.322884 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:41.325678 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:41.326290 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:41.326296 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:41.326304 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:41.326432 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:41.326777 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:41.326780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:41.326782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:41.327287 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:41.327547 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:41.327895 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 16:21:41.328281 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":189,"threads_started":1}
I20260806 16:21:41.332259 7674 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0325220a3445bf938655ffe31d0eec" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45971 } }
I20260806 16:21:41.332583 7674 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:41.332685 7674 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0325220a3445bf938655ffe31d0eec, State: Initialized, Role: FOLLOWER
I20260806 16:21:41.333230 7674 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0325220a3445bf938655ffe31d0eec" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45971 } }
I20260806 16:21:41.333420 7674 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:21:41.333513 7674 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:21:41.333648 7674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:41.334872 7674 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0325220a3445bf938655ffe31d0eec" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45971 } }
I20260806 16:21:41.335315 7674 leader_election.cc:302] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b0325220a3445bf938655ffe31d0eec; no voters:
I20260806 16:21:41.335783 7674 leader_election.cc:288] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:21:41.335980 7683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:41.337299 7683 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [term 1 LEADER]: Becoming Leader. State: Replica: 6b0325220a3445bf938655ffe31d0eec, State: Running, Role: LEADER
I20260806 16:21:41.337553 7674 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:21:41.338299 7683 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0325220a3445bf938655ffe31d0eec" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45971 } }
I20260806 16:21:41.340651 7685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b0325220a3445bf938655ffe31d0eec. Latest consensus state: current_term: 1 leader_uuid: "6b0325220a3445bf938655ffe31d0eec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0325220a3445bf938655ffe31d0eec" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45971 } } }
I20260806 16:21:41.340813 7685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:41.340791 7684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b0325220a3445bf938655ffe31d0eec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0325220a3445bf938655ffe31d0eec" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45971 } } }
I20260806 16:21:41.340900 7684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec [sys.catalog]: This master's current role is: LEADER
W20260806 16:21:41.343873 7702 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:21:41.343950 7702 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:21:41.344036 7703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:21:41.345347 7703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:21:41.354362 7703 catalog_manager.cc:1347] Generated new cluster ID: 62418f941d14410cb5a1a74544673366
I20260806 16:21:41.354498 7703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:21:41.379743 7703 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:21:41.381290 7703 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:21:41.389264 7703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec: Generated new TSK 0
I20260806 16:21:41.390290 7703 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.245.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
Entry for principal kudu/127.11.245.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab.
I20260806 16:21:41.436197 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:45971
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:41.621366 7710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:41.621783 7710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:41.621837 7710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:41.626956 7710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:41.627079 7710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:41.627153 7710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:21:41.634657 7710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.129.keytab
--principal=kudu/127.11.245.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:45971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:41.636683 7710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:41.640856 7710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:41.655332 7719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:41.655279 7716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:41.655282 7717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:41.655694 7710 server_base.cc:1034] running on GCE node
I20260806 16:21:41.656461 7710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:41.657888 7710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:41.659102 7710 hybrid_clock.cc:648] HybridClock initialized: now 1786033301659070 us; error 53 us; skew 500 ppm
Aug 06 16:21:41 dist-test-slave-t3q3 krb5kdc[7547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033301, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:41.663467 7710 init.cc:377] Logged in from keytab as kudu/127.11.245.129@KRBTEST.COM (short username kudu)
I20260806 16:21:41.664877 7710 webserver.cc:460] Webserver started at http://127.11.245.129:40521/ using document root <none> and password file <none>
I20260806 16:21:41.665666 7710 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:41.665738 7710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:41.666025 7710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:41.668883 7710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "38809de5188c4417a1129c0c2fc4e0e6"
format_stamp: "Formatted at 2026-08-06 16:21:41 on dist-test-slave-t3q3"
server_key: "bfbac19dd65ff52b71136e54c29dedf8"
server_key_iv: "9ddce6a95ecc9ac36ef08b96ab555bc0"
server_key_version: "encryptionkey@0"
I20260806 16:21:41.669658 7710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "38809de5188c4417a1129c0c2fc4e0e6"
format_stamp: "Formatted at 2026-08-06 16:21:41 on dist-test-slave-t3q3"
server_key: "bfbac19dd65ff52b71136e54c29dedf8"
server_key_iv: "9ddce6a95ecc9ac36ef08b96ab555bc0"
server_key_version: "encryptionkey@0"
I20260806 16:21:41.675043 7710 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:21:41.678659 7731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:41.681670 7710 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:21:41.681895 7710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "38809de5188c4417a1129c0c2fc4e0e6"
format_stamp: "Formatted at 2026-08-06 16:21:41 on dist-test-slave-t3q3"
server_key: "bfbac19dd65ff52b71136e54c29dedf8"
server_key_iv: "9ddce6a95ecc9ac36ef08b96ab555bc0"
server_key_version: "encryptionkey@0"
I20260806 16:21:41.682088 7710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:41.709371 7710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:41.714075 7710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:41.714385 7710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:41.715288 7710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:41.716529 7710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:41.716635 7710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:41.716722 7710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:41.716774 7710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:41.731645 7710 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:40551
I20260806 16:21:41.731954 7893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:40551 every 8 connection(s)
I20260806 16:21:41.733215 7710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:21:41.742033 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 7710
I20260806 16:21:41.742323 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:21:41.742861 12246 external_mini_cluster.cc:1428] Setting key 9590ebb7fc75df015b39447ee8b7c7d2
Aug 06 16:21:41 dist-test-slave-t3q3 krb5kdc[7547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033301, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.129@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:21:41.756302 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:41.736255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:49807 (local address 127.11.245.190:45971)
0806 16:21:41.736514 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:41.736521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:41.738627 (+ 2106us) server_negotiation.cc:407] Connection header received
0806 16:21:41.741039 (+ 2412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:41.741043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:41.741113 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:41.741207 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:41.743229 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:41.744343 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:41.746691 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:41.746987 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:41.747096 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:41.751044 (+ 3948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:41.751052 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:41.751059 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:41.751103 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:41.753262 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:41.754037 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:41.754044 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:41.754049 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:41.754162 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:41.754569 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:41.754575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:41.754579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:41.754942 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:41.755075 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:41.755827 (+ 752us) server_negotiation.cc:299] Negotiation successful
0806 16:21:41.756040 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 16:21:41.757472 7897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:41.737293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:45971 (local address 127.11.245.129:49807)
0806 16:21:41.738325 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:41.738383 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:41.740695 (+ 2312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:41.741576 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:41.741586 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:41.742072 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:41.743016 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:41.743037 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:41.744630 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:41.744635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:41.746500 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:41.746521 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:41.747200 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:41.747207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:41.747521 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:41.748458 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:21:41.748479 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:41.750760 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:41.753468 (+ 2708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:41.753479 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:41.753511 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:41.753876 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:41.754274 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:41.754277 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:41.754278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:41.754454 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:41.755116 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:41.755123 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:41.755502 (+ 379us) client_negotiation.cc:769] Sending connection context
0806 16:21:41.755726 (+ 224us) client_negotiation.cc:240] Negotiation successful
0806 16:21:41.756217 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":220,"threads_started":1}
WARNING: no policy specified for kudu/127.11.245.130@KRBTEST.COM; defaulting to no policy
I20260806 16:21:41.759267 7894 heartbeater.cc:347] Connected to a master server at 127.11.245.190:45971
I20260806 16:21:41.759842 7894 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:41.760676 7894 heartbeater.cc:511] Master 127.11.245.190:45971 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.130@KRBTEST.COM" created.
I20260806 16:21:41.763163 7613 ts_manager.cc:194] Registered new tserver with Master: 38809de5188c4417a1129c0c2fc4e0e6 (127.11.245.129:40551)
I20260806 16:21:41.764727 7613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.129@KRBTEST.COM'} at 127.11.245.129:49807
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
Entry for principal kudu/127.11.245.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab.
I20260806 16:21:41.790161 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:45971
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:41.949666 7900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:41.950035 7900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:41.950089 7900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:41.954792 7900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:41.954916 7900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:41.954991 7900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:21:41.962095 7900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.130.keytab
--principal=kudu/127.11.245.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:45971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:41.963737 7900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:41.965134 7900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:41.977896 7909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:41.977980 7900 server_base.cc:1034] running on GCE node
W20260806 16:21:41.977986 7906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:41.978049 7907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:41.978785 7900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:41.979697 7900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:41.980892 7900 hybrid_clock.cc:648] HybridClock initialized: now 1786033301980905 us; error 64 us; skew 500 ppm
Aug 06 16:21:41 dist-test-slave-t3q3 krb5kdc[7547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033301, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:41.984958 7900 init.cc:377] Logged in from keytab as kudu/127.11.245.130@KRBTEST.COM (short username kudu)
I20260806 16:21:41.986348 7900 webserver.cc:460] Webserver started at http://127.11.245.130:41409/ using document root <none> and password file <none>
I20260806 16:21:41.987135 7900 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:41.987222 7900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:41.987486 7900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:41.990620 7900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "3a3af12f8dbb462f9088fef50be5da76"
format_stamp: "Formatted at 2026-08-06 16:21:41 on dist-test-slave-t3q3"
server_key: "821ce4133dc838f68f93a18b913cebaf"
server_key_iv: "e77ccdbca4da134b0a02126ee4fd50e0"
server_key_version: "encryptionkey@0"
I20260806 16:21:41.991470 7900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "3a3af12f8dbb462f9088fef50be5da76"
format_stamp: "Formatted at 2026-08-06 16:21:41 on dist-test-slave-t3q3"
server_key: "821ce4133dc838f68f93a18b913cebaf"
server_key_iv: "e77ccdbca4da134b0a02126ee4fd50e0"
server_key_version: "encryptionkey@0"
I20260806 16:21:41.996922 7900 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:21:42.000299 7921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:42.002146 7900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:21:42.002370 7900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "3a3af12f8dbb462f9088fef50be5da76"
format_stamp: "Formatted at 2026-08-06 16:21:41 on dist-test-slave-t3q3"
server_key: "821ce4133dc838f68f93a18b913cebaf"
server_key_iv: "e77ccdbca4da134b0a02126ee4fd50e0"
server_key_version: "encryptionkey@0"
I20260806 16:21:42.002544 7900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:42.019152 7900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:42.024158 7900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:42.024485 7900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:42.025420 7900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:42.026598 7900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:42.026651 7900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:42.026729 7900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:42.026772 7900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:42.039076 7900 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:37211
I20260806 16:21:42.039119 8084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:37211 every 8 connection(s)
I20260806 16:21:42.040580 7900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:21:42.042949 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 7900
I20260806 16:21:42.043185 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:21:42.043685 12246 external_mini_cluster.cc:1428] Setting key a836ce3917e212dca5b98ba1bb16c185
Aug 06 16:21:42 dist-test-slave-t3q3 krb5kdc[7547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033301, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.130@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.245.131@KRBTEST.COM; defaulting to no policy
I20260806 16:21:42.060555 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.043476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:47647 (local address 127.11.245.190:45971)
0806 16:21:42.043686 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:42.043693 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:42.044752 (+ 1059us) server_negotiation.cc:407] Connection header received
0806 16:21:42.046465 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:42.046472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:42.046555 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:42.046681 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:42.048344 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.049264 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.050909 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.051154 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.051551 (+ 397us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.055578 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:42.055583 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:42.055585 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:42.055620 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:42.057481 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:42.058303 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:42.058307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:42.058308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:42.058399 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:42.058896 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:42.058903 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:42.058908 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:42.059171 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:42.059259 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:42.060189 (+ 930us) server_negotiation.cc:299] Negotiation successful
0806 16:21:42.060359 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.131@KRBTEST.COM" created.
I20260806 16:21:42.062177 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.043878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:45971 (local address 127.11.245.130:47647)
0806 16:21:42.044553 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:42.044617 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:42.046179 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:42.046928 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:42.046937 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:42.047400 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:42.048150 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:42.048177 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.049448 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.049451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:42.050711 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:42.050728 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.051564 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.051572 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:42.051854 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.052710 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:21:42.052743 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:42.055257 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:42.057646 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:42.057656 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:42.057685 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:42.058185 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:42.058517 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:42.058521 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:42.058522 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:42.058702 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:42.059315 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:42.059322 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:42.059785 (+ 463us) client_negotiation.cc:769] Sending connection context
0806 16:21:42.060035 (+ 250us) client_negotiation.cc:240] Negotiation successful
0806 16:21:42.060631 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":222,"threads_started":1}
I20260806 16:21:42.063982 8085 heartbeater.cc:347] Connected to a master server at 127.11.245.190:45971
I20260806 16:21:42.064512 8085 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:42.065435 8085 heartbeater.cc:511] Master 127.11.245.190:45971 requested a full tablet report, sending...
I20260806 16:21:42.067364 7613 ts_manager.cc:194] Registered new tserver with Master: 3a3af12f8dbb462f9088fef50be5da76 (127.11.245.130:37211)
I20260806 16:21:42.068243 7613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.130@KRBTEST.COM'} at 127.11.245.130:47647
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
Entry for principal kudu/127.11.245.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab.
I20260806 16:21:42.085462 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:45971
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:42.263250 8091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:42.263670 8091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:42.263724 8091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:42.268393 8091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:42.268483 8091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:42.268555 8091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:21:42.275159 8091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.131.keytab
--principal=kudu/127.11.245.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:45971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:42.277143 8091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:42.278692 8091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:42.291337 8103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:42.291430 8099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:42.291417 8100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:42.291914 8091 server_base.cc:1034] running on GCE node
I20260806 16:21:42.292550 8091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:42.293354 8091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:42.294546 8091 hybrid_clock.cc:648] HybridClock initialized: now 1786033302294510 us; error 58 us; skew 500 ppm
Aug 06 16:21:42 dist-test-slave-t3q3 krb5kdc[7547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033302, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:42.298317 8091 init.cc:377] Logged in from keytab as kudu/127.11.245.131@KRBTEST.COM (short username kudu)
I20260806 16:21:42.299813 8091 webserver.cc:460] Webserver started at http://127.11.245.131:39699/ using document root <none> and password file <none>
I20260806 16:21:42.300696 8091 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:42.300805 8091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:42.301113 8091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:42.304136 8091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "44bcf3e144bb4197bc6bac0ca32c2748"
format_stamp: "Formatted at 2026-08-06 16:21:42 on dist-test-slave-t3q3"
server_key: "b1837727aac6efd2ca5f8311eda133b2"
server_key_iv: "06dbda89124ae8cf6c4234d9f2015388"
server_key_version: "encryptionkey@0"
I20260806 16:21:42.304898 8091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "44bcf3e144bb4197bc6bac0ca32c2748"
format_stamp: "Formatted at 2026-08-06 16:21:42 on dist-test-slave-t3q3"
server_key: "b1837727aac6efd2ca5f8311eda133b2"
server_key_iv: "06dbda89124ae8cf6c4234d9f2015388"
server_key_version: "encryptionkey@0"
I20260806 16:21:42.310097 8091 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:21:42.313460 8112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:42.315243 8091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:21:42.315430 8091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "44bcf3e144bb4197bc6bac0ca32c2748"
format_stamp: "Formatted at 2026-08-06 16:21:42 on dist-test-slave-t3q3"
server_key: "b1837727aac6efd2ca5f8311eda133b2"
server_key_iv: "06dbda89124ae8cf6c4234d9f2015388"
server_key_version: "encryptionkey@0"
I20260806 16:21:42.315593 8091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:42.342079 8091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:42.346647 8091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:42.346935 8091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:42.347782 8091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:42.349005 8091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:42.349105 8091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:42.349215 8091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:42.349267 8091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:42.362114 8091 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:39351
I20260806 16:21:42.362138 8285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:39351 every 8 connection(s)
I20260806 16:21:42.363404 8091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:21:42.369508 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 8091
I20260806 16:21:42.369755 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:21:42.370162 12246 external_mini_cluster.cc:1428] Setting key 9ba95d0d80ecc5f8e075a93bc78b1998
Aug 06 16:21:42 dist-test-slave-t3q3 krb5kdc[7547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033302, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.131@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:21:42.383423 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.366059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:60071 (local address 127.11.245.190:45971)
0806 16:21:42.366265 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:42.366271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:42.368025 (+ 1754us) server_negotiation.cc:407] Connection header received
0806 16:21:42.369477 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:42.369481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:42.369557 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:42.369693 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:42.371616 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.372433 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.374090 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.374399 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.374500 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.378547 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:42.378560 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:42.378564 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:42.378605 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:42.380486 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:42.381377 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:42.381381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:42.381382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:42.381505 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:42.381937 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:42.381940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:42.381942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:42.382180 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:42.382268 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:42.383097 (+ 829us) server_negotiation.cc:299] Negotiation successful
0806 16:21:42.383253 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 16:21:42.385008 8288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.366579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:45971 (local address 127.11.245.131:60071)
0806 16:21:42.367755 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:42.367846 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:42.369112 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:42.369968 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:42.369978 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:42.370465 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:42.371232 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:42.371356 (+ 124us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.372588 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.372592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:42.373933 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:42.373945 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.374634 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.374637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:42.374813 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.375785 (+ 972us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:21:42.375816 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:42.378215 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:42.380843 (+ 2628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:42.380852 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:42.380876 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:42.381240 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:42.381640 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:21:42.381644 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:21:42.381645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:21:42.381833 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:21:42.382335 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:42.382342 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:42.382803 (+ 461us) client_negotiation.cc:769] Sending connection context
0806 16:21:42.383033 (+ 230us) client_negotiation.cc:240] Negotiation successful
0806 16:21:42.383431 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":896,"thread_start_us":636,"threads_started":1}
I20260806 16:21:42.386474 8286 heartbeater.cc:347] Connected to a master server at 127.11.245.190:45971
I20260806 16:21:42.386806 8286 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:42.387513 8286 heartbeater.cc:511] Master 127.11.245.190:45971 requested a full tablet report, sending...
I20260806 16:21:42.388989 7613 ts_manager.cc:194] Registered new tserver with Master: 44bcf3e144bb4197bc6bac0ca32c2748 (127.11.245.131:39351)
I20260806 16:21:42.390164 7613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.245.131@KRBTEST.COM'} at 127.11.245.131:60071
I20260806 16:21:42.398892 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:21:42.411583 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.401784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51260 (local address 127.11.245.190:45971)
0806 16:21:42.401989 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:42.401995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:42.402133 (+ 138us) server_negotiation.cc:407] Connection header received
0806 16:21:42.402329 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:42.402336 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:42.402402 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:42.402496 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:42.403336 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.403936 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.405183 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.405433 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.405557 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.407598 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:42.407602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:42.407604 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:42.407640 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:42.409526 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:42.410058 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:42.410067 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:42.410068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:42.410159 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:42.410465 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:42.410469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:42.410471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:42.410795 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:42.410959 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:42.411225 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 16:21:42.411375 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 16:21:42.416800 7613 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:21:42.421106 7613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:21:42.441242 8305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.433079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:37211 (local address 127.0.0.1:56232)
0806 16:21:42.434164 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:42.434182 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:42.434296 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:42.435364 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:42.435371 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:42.435419 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:42.435739 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:42.435751 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.437414 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.437420 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:42.438709 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:42.438722 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.440529 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.440532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:42.440658 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.440706 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 16:21:42.440838 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 16:21:42.441000 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1016,"mutex_wait_us":136,"spinlock_wait_cycles":25728,"thread_start_us":116,"threads_started":1}
I20260806 16:21:42.441306 8309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.434050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53350 (local address 127.11.245.129:40551)
0806 16:21:42.434948 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:42.434954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:42.434975 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:21:42.435046 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:42.435050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:42.435196 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:42.435386 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:42.436047 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.437192 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.438979 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.440273 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.440409 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.440564 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:42.440910 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 16:21:42.441059 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":776,"thread_start_us":148,"threads_started":1}
I20260806 16:21:42.441797 8303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.433034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:40551 (local address 127.0.0.1:53350)
0806 16:21:42.433496 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:42.433552 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:42.433701 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:42.435364 (+ 1663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:42.435371 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:42.435420 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:42.435909 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:42.435916 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.437334 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.437337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:42.438687 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:42.438771 (+ 84us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.440637 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.440640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:42.440730 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.440755 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 16:21:42.441550 (+ 795us) client_negotiation.cc:240] Negotiation successful
0806 16:21:42.441643 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":108,"spinlock_wait_cycles":4736,"thread_start_us":157,"threads_started":1}
I20260806 16:21:42.442027 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.434074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56232 (local address 127.11.245.130:37211)
0806 16:21:42.434625 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:42.434780 (+ 155us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:42.434811 (+ 31us) server_negotiation.cc:407] Connection header received
0806 16:21:42.434924 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:42.434931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:42.435184 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:42.435621 (+ 437us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:42.436008 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.437243 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.438888 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.440018 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.440213 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.440437 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:42.441735 (+ 1298us) server_negotiation.cc:299] Negotiation successful
0806 16:21:42.441861 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":163,"threads_started":1}
I20260806 16:21:42.442140 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.433366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:39351 (local address 127.0.0.1:40918)
0806 16:21:42.434527 (+ 1161us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:42.434541 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:42.434629 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:42.436475 (+ 1846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:42.436478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:42.436499 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:42.436590 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:42.436596 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.438286 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.438289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:42.439724 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:42.439759 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.441731 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.441734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:42.441830 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.441859 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 16:21:42.441936 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 16:21:42.442024 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1097,"thread_start_us":122,"threads_started":1}
I20260806 16:21:42.442584 8311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.434514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40918 (local address 127.11.245.131:39351)
0806 16:21:42.435930 (+ 1416us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:42.435946 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:42.435972 (+ 26us) server_negotiation.cc:407] Connection header received
0806 16:21:42.436090 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:42.436096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:42.436295 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:42.436467 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:42.436696 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.438129 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.439976 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.441377 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.441502 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.441632 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:42.442318 (+ 686us) server_negotiation.cc:299] Negotiation successful
0806 16:21:42.442425 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1294,"thread_start_us":203,"threads_started":1}
I20260806 16:21:42.445896 7803 tablet_service.cc:1467] Processing CreateTablet for tablet 0020f05b62b24cad897c9265acbb4c7f (DEFAULT_TABLE table=test-table [id=5a3c5fce237a45b5870dcb13015bee49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:42.445904 8188 tablet_service.cc:1467] Processing CreateTablet for tablet 0020f05b62b24cad897c9265acbb4c7f (DEFAULT_TABLE table=test-table [id=5a3c5fce237a45b5870dcb13015bee49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:42.445904 7995 tablet_service.cc:1467] Processing CreateTablet for tablet 0020f05b62b24cad897c9265acbb4c7f (DEFAULT_TABLE table=test-table [id=5a3c5fce237a45b5870dcb13015bee49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:42.447234 7803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0020f05b62b24cad897c9265acbb4c7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:42.447237 7995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0020f05b62b24cad897c9265acbb4c7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:42.447237 8188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0020f05b62b24cad897c9265acbb4c7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:42.456774 8312 tablet_bootstrap.cc:492] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6: Bootstrap starting.
I20260806 16:21:42.458389 8316 tablet_bootstrap.cc:492] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748: Bootstrap starting.
I20260806 16:21:42.459690 8312 tablet_bootstrap.cc:654] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:42.460563 8315 tablet_bootstrap.cc:492] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76: Bootstrap starting.
I20260806 16:21:42.460807 8312 log.cc:824] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:42.460911 8316 tablet_bootstrap.cc:654] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:42.462920 8316 log.cc:824] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:42.463383 8312 tablet_bootstrap.cc:492] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6: No bootstrap required, opened a new log
I20260806 16:21:42.463702 8312 ts_tablet_manager.cc:1397] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260806 16:21:42.464581 8315 tablet_bootstrap.cc:654] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:42.466857 8315 log.cc:824] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:42.467563 8316 tablet_bootstrap.cc:492] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748: No bootstrap required, opened a new log
I20260806 16:21:42.467864 8316 ts_tablet_manager.cc:1397] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 16:21:42.469321 8315 tablet_bootstrap.cc:492] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76: No bootstrap required, opened a new log
I20260806 16:21:42.469209 8312 raft_consensus.cc:348] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38809de5188c4417a1129c0c2fc4e0e6" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 40551 } } peers { permanent_uuid: "3a3af12f8dbb462f9088fef50be5da76" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37211 } } peers { permanent_uuid: "44bcf3e144bb4197bc6bac0ca32c2748" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39351 } }
I20260806 16:21:42.469594 8315 ts_tablet_manager.cc:1397] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260806 16:21:42.469616 8312 raft_consensus.cc:374] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:42.469717 8312 raft_consensus.cc:729] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38809de5188c4417a1129c0c2fc4e0e6, State: Initialized, Role: FOLLOWER
I20260806 16:21:42.470392 8312 consensus_queue.cc:260] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38809de5188c4417a1129c0c2fc4e0e6" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 40551 } } peers { permanent_uuid: "3a3af12f8dbb462f9088fef50be5da76" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37211 } } peers { permanent_uuid: "44bcf3e144bb4197bc6bac0ca32c2748" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39351 } }
I20260806 16:21:42.471349 7894 heartbeater.cc:503] Master 127.11.245.190:45971 was elected leader, sending a full tablet report...
I20260806 16:21:42.471895 8312 ts_tablet_manager.cc:1428] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 16:21:42.473201 8316 raft_consensus.cc:348] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38809de5188c4417a1129c0c2fc4e0e6" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 40551 } } peers { permanent_uuid: "3a3af12f8dbb462f9088fef50be5da76" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37211 } } peers { permanent_uuid: "44bcf3e144bb4197bc6bac0ca32c2748" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39351 } }
I20260806 16:21:42.473582 8316 raft_consensus.cc:374] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:42.473680 8316 raft_consensus.cc:729] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44bcf3e144bb4197bc6bac0ca32c2748, State: Initialized, Role: FOLLOWER
I20260806 16:21:42.474262 8316 consensus_queue.cc:260] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38809de5188c4417a1129c0c2fc4e0e6" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 40551 } } peers { permanent_uuid: "3a3af12f8dbb462f9088fef50be5da76" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37211 } } peers { permanent_uuid: "44bcf3e144bb4197bc6bac0ca32c2748" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39351 } }
I20260806 16:21:42.475109 8286 heartbeater.cc:503] Master 127.11.245.190:45971 was elected leader, sending a full tablet report...
I20260806 16:21:42.475564 8316 ts_tablet_manager.cc:1428] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 16:21:42.475605 8315 raft_consensus.cc:348] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38809de5188c4417a1129c0c2fc4e0e6" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 40551 } } peers { permanent_uuid: "3a3af12f8dbb462f9088fef50be5da76" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37211 } } peers { permanent_uuid: "44bcf3e144bb4197bc6bac0ca32c2748" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39351 } }
I20260806 16:21:42.475914 8315 raft_consensus.cc:374] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:42.476001 8315 raft_consensus.cc:729] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a3af12f8dbb462f9088fef50be5da76, State: Initialized, Role: FOLLOWER
I20260806 16:21:42.476562 8315 consensus_queue.cc:260] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38809de5188c4417a1129c0c2fc4e0e6" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 40551 } } peers { permanent_uuid: "3a3af12f8dbb462f9088fef50be5da76" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37211 } } peers { permanent_uuid: "44bcf3e144bb4197bc6bac0ca32c2748" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39351 } }
I20260806 16:21:42.477800 8085 heartbeater.cc:503] Master 127.11.245.190:45971 was elected leader, sending a full tablet report...
I20260806 16:21:42.478430 8315 ts_tablet_manager.cc:1428] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76: Time spent starting tablet: real 0.009s user 0.005s sys 0.001s
W20260806 16:21:42.485265 7895 tablet.cc:2369] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:21:42.492691 8322 raft_consensus.cc:482] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:21:42.493044 8322 raft_consensus.cc:504] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38809de5188c4417a1129c0c2fc4e0e6" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 40551 } } peers { permanent_uuid: "3a3af12f8dbb462f9088fef50be5da76" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37211 } } peers { permanent_uuid: "44bcf3e144bb4197bc6bac0ca32c2748" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39351 } }
I20260806 16:21:42.494673 8322 leader_election.cc:288] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a3af12f8dbb462f9088fef50be5da76 (127.11.245.130:37211), 44bcf3e144bb4197bc6bac0ca32c2748 (127.11.245.131:39351)
I20260806 16:21:42.499946 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.495232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:41597 (local address 127.11.245.130:37211)
0806 16:21:42.495462 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:42.495468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:42.495774 (+ 306us) server_negotiation.cc:407] Connection header received
0806 16:21:42.496171 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:42.496176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:42.496249 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:42.496358 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:42.496859 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.497487 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.498692 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.499145 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.499289 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.499373 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:42.499663 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 16:21:42.499750 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260806 16:21:42.500090 8327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.495113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:39351 (local address 127.11.245.129:44423)
0806 16:21:42.495770 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:42.495782 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:42.495913 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:42.496247 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:42.496250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:42.496266 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:42.496445 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:42.496452 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.497426 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.497432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:42.498537 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:42.498549 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.499283 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.499287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:42.499506 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.499554 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 16:21:42.499629 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 16:21:42.499782 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":205,"thread_start_us":164,"threads_started":1}
I20260806 16:21:42.500092 8326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.495115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:37211 (local address 127.11.245.129:41597)
0806 16:21:42.495651 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:42.495665 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:42.495885 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:42.496599 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:42.496601 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:42.496617 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:42.496733 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:42.496739 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.497637 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.497643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:42.498545 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:42.498556 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.499283 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.499287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:42.499473 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.499543 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 16:21:42.499629 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 16:21:42.499782 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":65,"thread_start_us":217,"threads_started":1}
I20260806 16:21:42.500461 8311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.495233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:44423 (local address 127.11.245.131:39351)
0806 16:21:42.495461 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:42.495467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:42.495900 (+ 433us) server_negotiation.cc:407] Connection header received
0806 16:21:42.496080 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:42.496083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:42.496139 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:42.496233 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:42.496560 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.497310 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.498677 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.499142 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.499283 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.499385 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:42.500218 (+ 833us) server_negotiation.cc:299] Negotiation successful
0806 16:21:42.500322 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 16:21:42.501253 8222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0020f05b62b24cad897c9265acbb4c7f" candidate_uuid: "38809de5188c4417a1129c0c2fc4e0e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44bcf3e144bb4197bc6bac0ca32c2748" is_pre_election: true
I20260806 16:21:42.501248 8025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0020f05b62b24cad897c9265acbb4c7f" candidate_uuid: "38809de5188c4417a1129c0c2fc4e0e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3af12f8dbb462f9088fef50be5da76" is_pre_election: true
I20260806 16:21:42.501739 8025 raft_consensus.cc:2454] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38809de5188c4417a1129c0c2fc4e0e6 in term 0.
I20260806 16:21:42.501809 8222 raft_consensus.cc:2454] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38809de5188c4417a1129c0c2fc4e0e6 in term 0.
I20260806 16:21:42.502440 7735 leader_election.cc:302] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38809de5188c4417a1129c0c2fc4e0e6, 3a3af12f8dbb462f9088fef50be5da76; no voters:
I20260806 16:21:42.502800 8322 raft_consensus.cc:2781] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:21:42.502907 8322 raft_consensus.cc:482] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:21:42.502961 8322 raft_consensus.cc:3037] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:42.504369 8322 raft_consensus.cc:504] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38809de5188c4417a1129c0c2fc4e0e6" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 40551 } } peers { permanent_uuid: "3a3af12f8dbb462f9088fef50be5da76" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37211 } } peers { permanent_uuid: "44bcf3e144bb4197bc6bac0ca32c2748" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39351 } }
I20260806 16:21:42.504844 8322 leader_election.cc:288] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [CANDIDATE]: Term 1 election: Requested vote from peers 3a3af12f8dbb462f9088fef50be5da76 (127.11.245.130:37211), 44bcf3e144bb4197bc6bac0ca32c2748 (127.11.245.131:39351)
I20260806 16:21:42.505347 8222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0020f05b62b24cad897c9265acbb4c7f" candidate_uuid: "38809de5188c4417a1129c0c2fc4e0e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44bcf3e144bb4197bc6bac0ca32c2748"
I20260806 16:21:42.505491 8025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0020f05b62b24cad897c9265acbb4c7f" candidate_uuid: "38809de5188c4417a1129c0c2fc4e0e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3af12f8dbb462f9088fef50be5da76"
I20260806 16:21:42.505558 8222 raft_consensus.cc:3037] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:42.505697 8025 raft_consensus.cc:3037] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:42.507047 8025 raft_consensus.cc:2454] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38809de5188c4417a1129c0c2fc4e0e6 in term 1.
I20260806 16:21:42.507052 8222 raft_consensus.cc:2454] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38809de5188c4417a1129c0c2fc4e0e6 in term 1.
I20260806 16:21:42.507422 7735 leader_election.cc:302] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38809de5188c4417a1129c0c2fc4e0e6, 3a3af12f8dbb462f9088fef50be5da76; no voters:
I20260806 16:21:42.507691 8322 raft_consensus.cc:2781] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:42.508011 8322 raft_consensus.cc:686] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [term 1 LEADER]: Becoming Leader. State: Replica: 38809de5188c4417a1129c0c2fc4e0e6, State: Running, Role: LEADER
I20260806 16:21:42.508587 8322 consensus_queue.cc:237] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38809de5188c4417a1129c0c2fc4e0e6" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 40551 } } peers { permanent_uuid: "3a3af12f8dbb462f9088fef50be5da76" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37211 } } peers { permanent_uuid: "44bcf3e144bb4197bc6bac0ca32c2748" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39351 } }
I20260806 16:21:42.513142 7613 catalog_manager.cc:5495] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38809de5188c4417a1129c0c2fc4e0e6 (127.11.245.129). New cstate: current_term: 1 leader_uuid: "38809de5188c4417a1129c0c2fc4e0e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38809de5188c4417a1129c0c2fc4e0e6" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 40551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a3af12f8dbb462f9088fef50be5da76" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44bcf3e144bb4197bc6bac0ca32c2748" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39351 } health_report { overall_health: UNKNOWN } } }
W20260806 16:21:42.542132 8086 tablet.cc:2369] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:21:42.546437 8309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.542047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53352 (local address 127.11.245.129:40551)
0806 16:21:42.542244 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:42.542250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:42.542271 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:21:42.542399 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:42.542402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:42.542492 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:42.542631 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:21:42.542992 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.543601 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.544615 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.544843 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.544932 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.545343 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:21:42.545993 (+ 650us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:21:42.546119 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:42.546218 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 16:21:42.546285 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 16:21:42.555442 8025 raft_consensus.cc:1261] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76 [term 1 FOLLOWER]: Refusing update from remote peer 38809de5188c4417a1129c0c2fc4e0e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:21:42.555442 8222 raft_consensus.cc:1261] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748 [term 1 FOLLOWER]: Refusing update from remote peer 38809de5188c4417a1129c0c2fc4e0e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:21:42.556528 8328 consensus_queue.cc:1035] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a3af12f8dbb462f9088fef50be5da76" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 37211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:21:42.557085 8322 consensus_queue.cc:1035] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44bcf3e144bb4197bc6bac0ca32c2748" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 39351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:21:42.583249 7612 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:21:42.583550 7612 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:21:42.587791 7612 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b0325220a3445bf938655ffe31d0eec: Sending DeleteTablet for 3 replicas of tablet 0020f05b62b24cad897c9265acbb4c7f
I20260806 16:21:42.589113 7995 tablet_service.cc:1514] Processing DeleteTablet for tablet 0020f05b62b24cad897c9265acbb4c7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:42 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:56232
I20260806 16:21:42.589116 7803 tablet_service.cc:1514] Processing DeleteTablet for tablet 0020f05b62b24cad897c9265acbb4c7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:42 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:53350
I20260806 16:21:42.589522 8188 tablet_service.cc:1514] Processing DeleteTablet for tablet 0020f05b62b24cad897c9265acbb4c7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:42 UTC) from {username='kudu', principal='kudu/127.11.245.190@KRBTEST.COM'} at 127.0.0.1:40918
I20260806 16:21:42.589885 8347 tablet_replica.cc:330] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76: stopping tablet replica
I20260806 16:21:42.590358 8347 raft_consensus.cc:2229] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:21:42.590416 8349 tablet_replica.cc:330] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748: stopping tablet replica
I20260806 16:21:42.590771 8349 raft_consensus.cc:2229] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:21:42.590852 8347 raft_consensus.cc:2258] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:21:42.591354 8349 raft_consensus.cc:2258] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:21:42.591462 8346 tablet_replica.cc:330] T 0020f05b62b24cad897c9265acbb4c7f P 38809de5188c4417a1129c0c2fc4e0e6: stopping tablet replica
I20260806 16:21:42.591536 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 7710
I20260806 16:21:42.599341 8347 ts_tablet_manager.cc:1905] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:21:42.601125 8349 ts_tablet_manager.cc:1905] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:21:42.603961 8349 ts_tablet_manager.cc:1918] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:21:42.604158 8349 log.cc:1196] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/0020f05b62b24cad897c9265acbb4c7f
I20260806 16:21:42.604290 8347 ts_tablet_manager.cc:1918] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:21:42.604444 8347 log.cc:1196] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/0020f05b62b24cad897c9265acbb4c7f
I20260806 16:21:42.604655 8349 ts_tablet_manager.cc:1939] T 0020f05b62b24cad897c9265acbb4c7f P 44bcf3e144bb4197bc6bac0ca32c2748: Deleting consensus metadata
I20260806 16:21:42.604847 8347 ts_tablet_manager.cc:1939] T 0020f05b62b24cad897c9265acbb4c7f P 3a3af12f8dbb462f9088fef50be5da76: Deleting consensus metadata
I20260806 16:21:42.606988 7590 catalog_manager.cc:4841] TS 44bcf3e144bb4197bc6bac0ca32c2748 (127.11.245.131:39351): tablet 0020f05b62b24cad897c9265acbb4c7f (table test-table [id=5a3c5fce237a45b5870dcb13015bee49]) successfully deleted
I20260806 16:21:42.606990 7592 catalog_manager.cc:4841] TS 3a3af12f8dbb462f9088fef50be5da76 (127.11.245.130:37211): tablet 0020f05b62b24cad897c9265acbb4c7f (table test-table [id=5a3c5fce237a45b5870dcb13015bee49]) successfully deleted
W20260806 16:21:42.611176 7593 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:40551 (error 108)
I20260806 16:21:42.611337 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 7900
I20260806 16:21:42.621915 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.621209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:40551 (local address 127.0.0.1:53356)
0806 16:21:42.621635 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:42.621747 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:40551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279}
W20260806 16:21:42.622216 7593 catalog_manager.cc:4584] TS 38809de5188c4417a1129c0c2fc4e0e6 (127.11.245.129:40551): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0020f05b62b24cad897c9265acbb4c7f: Network error: Client connection negotiation failed: client connection to 127.11.245.129:40551: connect: Connection refused (error 111)
I20260806 16:21:42.631053 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 8091
I20260806 16:21:42.651549 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 7566
2026-08-06T16:21:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (3083 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T16:21:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:21:42Z Disabled control of system clock
I20260806 16:21:42.704043 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:45285
--webserver_interface=127.11.245.190
--webserver_port=0
--builtin_ntp_servers=127.11.245.148:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:45285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:21:42.872573 8361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:42.873001 8361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:42.873067 8361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:42.879662 8361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:21:42.879809 8361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:42.879840 8361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:42.879868 8361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:21:42.879892 8361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:21:42.886651 8361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:45285
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:45285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:42.888365 8361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:42.890237 8361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:42.900946 8368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:42.900995 8367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:42.901005 8371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:42.901667 8361 server_base.cc:1034] running on GCE node
I20260806 16:21:42.902383 8361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:42.903903 8361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:42.905135 8361 hybrid_clock.cc:648] HybridClock initialized: now 1786033302905082 us; error 65 us; skew 500 ppm
I20260806 16:21:42.907696 8361 webserver.cc:460] Webserver started at http://127.11.245.190:35333/ using document root <none> and password file <none>
I20260806 16:21:42.908567 8361 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:42.908679 8361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:42.908983 8361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:42.911772 8361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "f13c69a545c041b08b63fa04912a4ac5"
format_stamp: "Formatted at 2026-08-06 16:21:42 on dist-test-slave-t3q3"
server_key: "7c5423d1603cbf91c8a9dc6afb2ffb37"
server_key_iv: "f5aa906d795d0d7661b6d601213b529c"
server_key_version: "encryptionkey@0"
I20260806 16:21:42.912909 8361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "f13c69a545c041b08b63fa04912a4ac5"
format_stamp: "Formatted at 2026-08-06 16:21:42 on dist-test-slave-t3q3"
server_key: "7c5423d1603cbf91c8a9dc6afb2ffb37"
server_key_iv: "f5aa906d795d0d7661b6d601213b529c"
server_key_version: "encryptionkey@0"
I20260806 16:21:42.918911 8361 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 16:21:42.922704 8380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:42.924685 8361 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:21:42.924916 8361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "f13c69a545c041b08b63fa04912a4ac5"
format_stamp: "Formatted at 2026-08-06 16:21:42 on dist-test-slave-t3q3"
server_key: "7c5423d1603cbf91c8a9dc6afb2ffb37"
server_key_iv: "f5aa906d795d0d7661b6d601213b529c"
server_key_version: "encryptionkey@0"
I20260806 16:21:42.925098 8361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:42.939411 8361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:42.940567 8361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:42.940824 8361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:42.950644 8361 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:45285
I20260806 16:21:42.950649 8456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:45285 every 8 connection(s)
I20260806 16:21:42.951963 8361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:21:42.954838 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 8361
I20260806 16:21:42.955036 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:21:42.955439 12246 external_mini_cluster.cc:1428] Setting key 567e09fb4a1695bbe283f640d105d11d
I20260806 16:21:42.957501 8460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:42.966507 8460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5: Bootstrap starting.
I20260806 16:21:42.969033 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:42.957616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43934 (local address 127.11.245.190:45285)
0806 16:21:42.958789 (+ 1173us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:42.958810 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:42.958870 (+ 60us) server_negotiation.cc:407] Connection header received
0806 16:21:42.959697 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:42.959720 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:42.960493 (+ 773us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:42.961058 (+ 565us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:42.961828 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.963021 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.964259 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:42.964815 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:42.965124 (+ 309us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:42.966654 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:42.966660 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:42.966679 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:42.967248 (+ 569us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:42.967320 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:42.967346 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:42.967476 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:42.967637 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 16:21:42.967997 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":209,"threads_started":1}
I20260806 16:21:42.969933 8460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:42.970968 8460 log.cc:824] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:42.973675 8460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5: No bootstrap required, opened a new log
I20260806 16:21:42.978693 8460 raft_consensus.cc:348] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13c69a545c041b08b63fa04912a4ac5" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45285 } }
I20260806 16:21:42.979329 8460 raft_consensus.cc:374] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:42.979480 8460 raft_consensus.cc:729] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f13c69a545c041b08b63fa04912a4ac5, State: Initialized, Role: FOLLOWER
I20260806 16:21:42.980172 8460 consensus_queue.cc:260] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13c69a545c041b08b63fa04912a4ac5" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45285 } }
I20260806 16:21:42.980346 8460 raft_consensus.cc:388] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:21:42.980427 8460 raft_consensus.cc:482] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:21:42.980572 8460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:42.982076 8460 raft_consensus.cc:504] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13c69a545c041b08b63fa04912a4ac5" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45285 } }
I20260806 16:21:42.982613 8460 leader_election.cc:302] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f13c69a545c041b08b63fa04912a4ac5; no voters:
I20260806 16:21:42.983116 8460 leader_election.cc:288] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:21:42.983394 8471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:42.983726 8471 raft_consensus.cc:686] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [term 1 LEADER]: Becoming Leader. State: Replica: f13c69a545c041b08b63fa04912a4ac5, State: Running, Role: LEADER
I20260806 16:21:42.984522 8471 consensus_queue.cc:237] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13c69a545c041b08b63fa04912a4ac5" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45285 } }
I20260806 16:21:42.985237 8460 sys_catalog.cc:564] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:21:42.987048 8474 sys_catalog.cc:455] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f13c69a545c041b08b63fa04912a4ac5. Latest consensus state: current_term: 1 leader_uuid: "f13c69a545c041b08b63fa04912a4ac5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13c69a545c041b08b63fa04912a4ac5" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45285 } } }
I20260806 16:21:42.987221 8474 sys_catalog.cc:458] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:42.987931 8473 sys_catalog.cc:455] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f13c69a545c041b08b63fa04912a4ac5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13c69a545c041b08b63fa04912a4ac5" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45285 } } }
I20260806 16:21:42.988106 8473 sys_catalog.cc:458] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5 [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:42.988440 8482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:21:42.992511 8482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:21:43.001096 8482 catalog_manager.cc:1347] Generated new cluster ID: 48c1cbf9209b4f6da36f27e5f7390591
I20260806 16:21:43.001202 8482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:21:43.012281 8482 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:21:43.013639 8482 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:21:43.021272 8482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5: Generated new TSK 0
I20260806 16:21:43.022507 8482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:21:43.026082 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.129:0
--local_ip_for_outbound_sockets=127.11.245.129
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:45285
--builtin_ntp_servers=127.11.245.148:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:21:43.173969 8494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:43.174435 8494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:43.174530 8494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:43.179365 8494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:43.179523 8494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:43.179644 8494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.129
I20260806 16:21:43.186123 8494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.245.129
--webserver_port=0
--tserver_master_addrs=127.11.245.190:45285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.129
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:43.188009 8494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:43.190079 8494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:43.203689 8502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:43.203806 8494 server_base.cc:1034] running on GCE node
W20260806 16:21:43.203823 8501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:43.203729 8505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:43.204603 8494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:43.205505 8494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:43.206761 8494 hybrid_clock.cc:648] HybridClock initialized: now 1786033303206719 us; error 39 us; skew 500 ppm
I20260806 16:21:43.209323 8494 webserver.cc:460] Webserver started at http://127.11.245.129:34417/ using document root <none> and password file <none>
I20260806 16:21:43.210107 8494 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:43.210184 8494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:43.210460 8494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:43.213253 8494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/data/instance:
uuid: "91d70ac561d54da1ababa2a04b7582be"
format_stamp: "Formatted at 2026-08-06 16:21:43 on dist-test-slave-t3q3"
server_key: "ef2c5e4e3a4a4568ddba9983dff79312"
server_key_iv: "057a99ad87c7c027488189f33dde0e29"
server_key_version: "encryptionkey@0"
I20260806 16:21:43.213946 8494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance:
uuid: "91d70ac561d54da1ababa2a04b7582be"
format_stamp: "Formatted at 2026-08-06 16:21:43 on dist-test-slave-t3q3"
server_key: "ef2c5e4e3a4a4568ddba9983dff79312"
server_key_iv: "057a99ad87c7c027488189f33dde0e29"
server_key_version: "encryptionkey@0"
I20260806 16:21:43.218770 8494 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:21:43.222644 8517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:43.224481 8494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:21:43.224644 8494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/wal
uuid: "91d70ac561d54da1ababa2a04b7582be"
format_stamp: "Formatted at 2026-08-06 16:21:43 on dist-test-slave-t3q3"
server_key: "ef2c5e4e3a4a4568ddba9983dff79312"
server_key_iv: "057a99ad87c7c027488189f33dde0e29"
server_key_version: "encryptionkey@0"
I20260806 16:21:43.224797 8494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:43.245430 8494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:43.246534 8494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:43.246815 8494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:43.247721 8494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:43.249120 8494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:43.249177 8494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:43.249261 8494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:43.249295 8494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:43.262588 8494 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.129:45077
I20260806 16:21:43.262699 8702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.129:45077 every 8 connection(s)
I20260806 16:21:43.263901 8494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/data/info.pb
I20260806 16:21:43.268429 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 8494
I20260806 16:21:43.268587 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-0/wal/instance
I20260806 16:21:43.268959 12246 external_mini_cluster.cc:1428] Setting key c506746410606f42f790b3a9f5ddb938
I20260806 16:21:43.271836 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.130:0
--local_ip_for_outbound_sockets=127.11.245.130
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:45285
--builtin_ntp_servers=127.11.245.148:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:21:43.278028 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.266534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:44113 (local address 127.11.245.190:45285)
0806 16:21:43.266743 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:43.266749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:43.268243 (+ 1494us) server_negotiation.cc:407] Connection header received
0806 16:21:43.270209 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:43.270213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:43.270277 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:43.270423 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:43.271807 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.272725 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.275052 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.275457 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.275631 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.277270 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:43.277277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:43.277283 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:43.277308 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:43.277341 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:43.277356 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:43.277461 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:43.277653 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 16:21:43.277766 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 16:21:43.279289 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.267068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:45285 (local address 127.11.245.129:44113)
0806 16:21:43.267962 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:43.268014 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:43.269958 (+ 1944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:43.270669 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:43.270678 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:43.270838 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:43.271600 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.271617 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.272921 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.272958 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.274889 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.274904 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.275739 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.275742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.275972 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.276868 (+ 896us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:21:43.276870 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:43.276945 (+ 75us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:21:43.277046 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:43.277466 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:43.277476 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:43.277483 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 16:21:43.277625 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 16:21:43.277969 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":151,"threads_started":1}
I20260806 16:21:43.281123 8703 heartbeater.cc:347] Connected to a master server at 127.11.245.190:45285
I20260806 16:21:43.281456 8703 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:43.282198 8703 heartbeater.cc:511] Master 127.11.245.190:45285 requested a full tablet report, sending...
I20260806 16:21:43.284503 8406 ts_manager.cc:194] Registered new tserver with Master: 91d70ac561d54da1ababa2a04b7582be (127.11.245.129:45077)
I20260806 16:21:43.286710 8406 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.245.129:44113
W20260806 16:21:43.424410 8706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:43.424783 8706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:43.424829 8706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:43.429082 8706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:43.429157 8706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:43.429220 8706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.130
I20260806 16:21:43.434782 8706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.245.130
--webserver_port=0
--tserver_master_addrs=127.11.245.190:45285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.130
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:43.436367 8706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:43.438190 8706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:43.450865 8728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:43.450906 8716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:43.450867 8720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:43.451413 8706 server_base.cc:1034] running on GCE node
I20260806 16:21:43.451952 8706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:43.452898 8706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:43.454145 8706 hybrid_clock.cc:648] HybridClock initialized: now 1786033303454115 us; error 41 us; skew 500 ppm
I20260806 16:21:43.456881 8706 webserver.cc:460] Webserver started at http://127.11.245.130:38831/ using document root <none> and password file <none>
I20260806 16:21:43.457677 8706 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:43.457754 8706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:43.458091 8706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:43.460857 8706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/data/instance:
uuid: "86abc4ab6aa5458ab58bf64e0a8240a5"
format_stamp: "Formatted at 2026-08-06 16:21:43 on dist-test-slave-t3q3"
server_key: "5fa75d25d70f336f874532afe0e0e764"
server_key_iv: "097cb5f689b21303f0ec6491209af565"
server_key_version: "encryptionkey@0"
I20260806 16:21:43.461594 8706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance:
uuid: "86abc4ab6aa5458ab58bf64e0a8240a5"
format_stamp: "Formatted at 2026-08-06 16:21:43 on dist-test-slave-t3q3"
server_key: "5fa75d25d70f336f874532afe0e0e764"
server_key_iv: "097cb5f689b21303f0ec6491209af565"
server_key_version: "encryptionkey@0"
I20260806 16:21:43.466645 8706 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:21:43.469928 8738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:43.471735 8706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:21:43.471920 8706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/wal
uuid: "86abc4ab6aa5458ab58bf64e0a8240a5"
format_stamp: "Formatted at 2026-08-06 16:21:43 on dist-test-slave-t3q3"
server_key: "5fa75d25d70f336f874532afe0e0e764"
server_key_iv: "097cb5f689b21303f0ec6491209af565"
server_key_version: "encryptionkey@0"
I20260806 16:21:43.472100 8706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:43.494194 8706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:43.495473 8706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:43.495762 8706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:43.496851 8706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:43.498184 8706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:43.498270 8706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:43.498382 8706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:43.498520 8706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:43.511021 8706 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.130:32801
I20260806 16:21:43.511059 8916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.130:32801 every 8 connection(s)
I20260806 16:21:43.512383 8706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/data/info.pb
I20260806 16:21:43.514532 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 8706
I20260806 16:21:43.514706 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/wal/instance
I20260806 16:21:43.515177 12246 external_mini_cluster.cc:1428] Setting key 758d770ffd251945ad6f1885cacacd4e
I20260806 16:21:43.517616 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.245.131:0
--local_ip_for_outbound_sockets=127.11.245.131
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:45285
--builtin_ntp_servers=127.11.245.148:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:21:43.525416 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.514982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:59345 (local address 127.11.245.190:45285)
0806 16:21:43.515171 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:43.515176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:43.516807 (+ 1631us) server_negotiation.cc:407] Connection header received
0806 16:21:43.518677 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:43.518682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:43.518749 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:43.518839 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:43.520023 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.520846 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.522553 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.522949 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.523084 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.524605 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:43.524613 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:43.524618 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:43.524643 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:43.524675 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:43.524690 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:43.524790 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:43.525027 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 16:21:43.525135 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 16:21:43.526643 8921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.515651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:45285 (local address 127.11.245.130:59345)
0806 16:21:43.516618 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:43.516686 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:43.518404 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:43.519076 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:43.519085 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:43.519195 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:43.519860 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.519876 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.521026 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.521033 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.522390 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.522404 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.523146 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.523149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.523338 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.524256 (+ 918us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:21:43.524258 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:43.524312 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:21:43.524404 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:43.524829 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:43.524840 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:43.524847 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 16:21:43.524998 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 16:21:43.525320 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":185,"threads_started":1}
I20260806 16:21:43.528620 8917 heartbeater.cc:347] Connected to a master server at 127.11.245.190:45285
I20260806 16:21:43.529026 8917 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:43.530088 8917 heartbeater.cc:511] Master 127.11.245.190:45285 requested a full tablet report, sending...
I20260806 16:21:43.532232 8406 ts_manager.cc:194] Registered new tserver with Master: 86abc4ab6aa5458ab58bf64e0a8240a5 (127.11.245.130:32801)
I20260806 16:21:43.533140 8406 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.245.130:59345
W20260806 16:21:43.674535 8923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:43.674953 8923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:43.675002 8923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:43.679908 8923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:43.680001 8923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:43.680142 8923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.245.131
I20260806 16:21:43.685850 8923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.245.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.245.131
--webserver_port=0
--tserver_master_addrs=127.11.245.190:45285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.245.131
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:43.687609 8923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:43.689584 8923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:43.703408 8935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:43.703614 8933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:43.703518 8937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:43.703781 8923 server_base.cc:1034] running on GCE node
I20260806 16:21:43.704569 8923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:43.705592 8923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:43.706816 8923 hybrid_clock.cc:648] HybridClock initialized: now 1786033303706779 us; error 64 us; skew 500 ppm
I20260806 16:21:43.709537 8923 webserver.cc:460] Webserver started at http://127.11.245.131:37543/ using document root <none> and password file <none>
I20260806 16:21:43.710386 8923 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:43.710512 8923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:43.710829 8923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:43.714926 8923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/data/instance:
uuid: "5ab9b281b34e4ea48c732bdd766282a0"
format_stamp: "Formatted at 2026-08-06 16:21:43 on dist-test-slave-t3q3"
server_key: "571c1ca16e1e3427b23526727125fe8a"
server_key_iv: "7f9b43f2c32b9810977cc22f144c4a50"
server_key_version: "encryptionkey@0"
I20260806 16:21:43.716601 8923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance:
uuid: "5ab9b281b34e4ea48c732bdd766282a0"
format_stamp: "Formatted at 2026-08-06 16:21:43 on dist-test-slave-t3q3"
server_key: "571c1ca16e1e3427b23526727125fe8a"
server_key_iv: "7f9b43f2c32b9810977cc22f144c4a50"
server_key_version: "encryptionkey@0"
I20260806 16:21:43.722363 8923 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:21:43.726508 8947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:43.728680 8923 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:21:43.728884 8923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/wal
uuid: "5ab9b281b34e4ea48c732bdd766282a0"
format_stamp: "Formatted at 2026-08-06 16:21:43 on dist-test-slave-t3q3"
server_key: "571c1ca16e1e3427b23526727125fe8a"
server_key_iv: "7f9b43f2c32b9810977cc22f144c4a50"
server_key_version: "encryptionkey@0"
I20260806 16:21:43.729082 8923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:43.754117 8923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:43.755204 8923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:43.755479 8923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:43.756347 8923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:21:43.757659 8923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:21:43.757741 8923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:43.757848 8923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:21:43.757899 8923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:43.771267 8923 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.131:46531
I20260806 16:21:43.771287 9108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.131:46531 every 8 connection(s)
I20260806 16:21:43.772694 8923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/data/info.pb
I20260806 16:21:43.780405 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 8923
I20260806 16:21:43.780581 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/wal/instance
I20260806 16:21:43.780964 12246 external_mini_cluster.cc:1428] Setting key 7d36368b44341e0d981f0c585b0fd4a0
I20260806 16:21:43.786541 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.775513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.131:41475 (local address 127.11.245.190:45285)
0806 16:21:43.776038 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:43.776043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:43.777412 (+ 1369us) server_negotiation.cc:407] Connection header received
0806 16:21:43.779239 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:43.779243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:43.779325 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:43.779423 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:43.780992 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.781677 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.783688 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.784026 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.784193 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.785757 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:43.785761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:43.785763 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:43.785779 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:43.785804 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:43.785813 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:43.785923 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:43.786130 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 16:21:43.786293 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293}
I20260806 16:21:43.787912 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.776084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:45285 (local address 127.11.245.131:41475)
0806 16:21:43.777189 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:43.777244 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:43.778921 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:43.779731 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:43.779740 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:43.779905 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:21:43.780796 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.780818 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.781864 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.781868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.783459 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.783476 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.784255 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.784260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.784471 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.785309 (+ 838us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:21:43.785311 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:21:43.785444 (+ 133us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:21:43.785552 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:21:43.785927 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:21:43.785941 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:21:43.785952 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 16:21:43.786122 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 16:21:43.786721 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":175,"threads_started":1}
I20260806 16:21:43.789508 9110 heartbeater.cc:347] Connected to a master server at 127.11.245.190:45285
I20260806 16:21:43.789875 9110 heartbeater.cc:464] Registering TS with master...
I20260806 16:21:43.790659 9110 heartbeater.cc:511] Master 127.11.245.190:45285 requested a full tablet report, sending...
I20260806 16:21:43.792362 8406 ts_manager.cc:194] Registered new tserver with Master: 5ab9b281b34e4ea48c732bdd766282a0 (127.11.245.131:46531)
I20260806 16:21:43.793079 8406 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.245.131:41475
I20260806 16:21:43.796777 12246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:21:43.804467 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.799610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43938 (local address 127.11.245.190:45285)
0806 16:21:43.799836 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:43.799842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:43.800034 (+ 192us) server_negotiation.cc:407] Connection header received
0806 16:21:43.800274 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:43.800277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:43.800353 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:43.800450 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:43.800790 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.801385 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.802318 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.802554 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.802669 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.803846 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:43.803850 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:43.803853 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:43.803869 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:43.803894 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:43.803904 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:43.804002 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:43.804201 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 16:21:43.804277 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260806 16:21:43.809223 8406 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:21:43.814066 8406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:21:43.835156 9135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.829390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41400 (local address 127.11.245.129:45077)
0806 16:21:43.829932 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:43.830016 (+ 84us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:43.830047 (+ 31us) server_negotiation.cc:407] Connection header received
0806 16:21:43.830149 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:43.830155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:43.830388 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:43.830893 (+ 505us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:43.831101 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.832161 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.833502 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.834392 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.834542 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.834737 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:43.834800 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 16:21:43.835007 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":137,"threads_started":1}
I20260806 16:21:43.835130 9129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.827021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:46531 (local address 127.0.0.1:52290)
0806 16:21:43.828576 (+ 1555us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:43.828591 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:43.828698 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:43.829471 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:43.829475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:43.829495 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:43.829943 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.829955 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.831495 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.831499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.832744 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.832755 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.834340 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.834345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.834692 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.834721 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 16:21:43.834779 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 16:21:43.834944 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1482,"mutex_wait_us":225,"thread_start_us":118,"threads_started":1}
I20260806 16:21:43.835132 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.829136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:45077 (local address 127.0.0.1:41400)
0806 16:21:43.829495 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:43.829510 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:43.829601 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:43.830594 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:43.830599 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:43.830620 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:43.830720 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.830726 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.832290 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.832293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.833327 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.833358 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.834529 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.834533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.834618 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.834641 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 16:21:43.834708 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 16:21:43.834896 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":28,"thread_start_us":119,"threads_started":1}
I20260806 16:21:43.835301 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.827155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52612 (local address 127.11.245.130:32801)
0806 16:21:43.827885 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:43.827909 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:43.828360 (+ 451us) server_negotiation.cc:407] Connection header received
0806 16:21:43.828624 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:43.828635 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:43.828996 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:43.829189 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:43.829945 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.831401 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.833121 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.834033 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.834160 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.834317 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:43.834890 (+ 573us) server_negotiation.cc:299] Negotiation successful
0806 16:21:43.835085 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":254,"threads_started":1}
I20260806 16:21:43.835724 9128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.826954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:32801 (local address 127.0.0.1:52612)
0806 16:21:43.828054 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:43.828151 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:43.828353 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:43.829434 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:43.829442 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:43.829585 (+ 143us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:43.829777 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.829787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.831558 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.831565 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.832958 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.832970 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.834369 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.834372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.834527 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.834575 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 16:21:43.834708 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 16:21:43.835619 (+ 911us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":868,"mutex_wait_us":811,"spinlock_wait_cycles":896,"thread_start_us":138,"threads_started":1}
I20260806 16:21:43.836383 9133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.828017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52290 (local address 127.11.245.131:46531)
0806 16:21:43.828540 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:43.828554 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:43.828699 (+ 145us) server_negotiation.cc:407] Connection header received
0806 16:21:43.828819 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:43.828825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:43.829101 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:43.829292 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:43.830098 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.831347 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.832890 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.833808 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.833993 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.834263 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:43.836022 (+ 1759us) server_negotiation.cc:299] Negotiation successful
0806 16:21:43.836188 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":230,"threads_started":1}
I20260806 16:21:43.839963 8600 tablet_service.cc:1467] Processing CreateTablet for tablet 57570d979f8b4322bab140cd10bec521 (DEFAULT_TABLE table=test-table [id=63f23f9089ad4c25b526f592e119e252]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:43.839972 9021 tablet_service.cc:1467] Processing CreateTablet for tablet 57570d979f8b4322bab140cd10bec521 (DEFAULT_TABLE table=test-table [id=63f23f9089ad4c25b526f592e119e252]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:43.839962 8815 tablet_service.cc:1467] Processing CreateTablet for tablet 57570d979f8b4322bab140cd10bec521 (DEFAULT_TABLE table=test-table [id=63f23f9089ad4c25b526f592e119e252]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:21:43.841483 8600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57570d979f8b4322bab140cd10bec521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:43.841482 9021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57570d979f8b4322bab140cd10bec521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:43.841483 8815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57570d979f8b4322bab140cd10bec521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:43.850986 9138 tablet_bootstrap.cc:492] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be: Bootstrap starting.
I20260806 16:21:43.852977 9142 tablet_bootstrap.cc:492] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5: Bootstrap starting.
I20260806 16:21:43.854647 9138 tablet_bootstrap.cc:654] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:43.855418 9142 tablet_bootstrap.cc:654] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:43.855759 9138 log.cc:824] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:43.856676 9142 log.cc:824] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:43.856772 9141 tablet_bootstrap.cc:492] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0: Bootstrap starting.
I20260806 16:21:43.859045 9138 tablet_bootstrap.cc:492] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be: No bootstrap required, opened a new log
I20260806 16:21:43.859359 9138 ts_tablet_manager.cc:1397] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 16:21:43.859752 9142 tablet_bootstrap.cc:492] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5: No bootstrap required, opened a new log
I20260806 16:21:43.859977 9141 tablet_bootstrap.cc:654] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:43.860004 9142 ts_tablet_manager.cc:1397] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 16:21:43.861102 9141 log.cc:824] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:43.864151 9141 tablet_bootstrap.cc:492] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0: No bootstrap required, opened a new log
I20260806 16:21:43.864444 9141 ts_tablet_manager.cc:1397] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 16:21:43.864293 9142 raft_consensus.cc:348] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86abc4ab6aa5458ab58bf64e0a8240a5" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 32801 } } peers { permanent_uuid: "5ab9b281b34e4ea48c732bdd766282a0" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 46531 } } peers { permanent_uuid: "91d70ac561d54da1ababa2a04b7582be" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45077 } }
I20260806 16:21:43.864684 9142 raft_consensus.cc:374] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:43.864748 9142 raft_consensus.cc:729] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86abc4ab6aa5458ab58bf64e0a8240a5, State: Initialized, Role: FOLLOWER
I20260806 16:21:43.865037 9138 raft_consensus.cc:348] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86abc4ab6aa5458ab58bf64e0a8240a5" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 32801 } } peers { permanent_uuid: "5ab9b281b34e4ea48c732bdd766282a0" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 46531 } } peers { permanent_uuid: "91d70ac561d54da1ababa2a04b7582be" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45077 } }
I20260806 16:21:43.865356 9138 raft_consensus.cc:374] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:43.865438 9138 raft_consensus.cc:729] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91d70ac561d54da1ababa2a04b7582be, State: Initialized, Role: FOLLOWER
I20260806 16:21:43.865372 9142 consensus_queue.cc:260] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86abc4ab6aa5458ab58bf64e0a8240a5" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 32801 } } peers { permanent_uuid: "5ab9b281b34e4ea48c732bdd766282a0" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 46531 } } peers { permanent_uuid: "91d70ac561d54da1ababa2a04b7582be" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45077 } }
I20260806 16:21:43.865931 9138 consensus_queue.cc:260] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86abc4ab6aa5458ab58bf64e0a8240a5" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 32801 } } peers { permanent_uuid: "5ab9b281b34e4ea48c732bdd766282a0" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 46531 } } peers { permanent_uuid: "91d70ac561d54da1ababa2a04b7582be" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45077 } }
I20260806 16:21:43.867161 9138 ts_tablet_manager.cc:1428] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be: Time spent starting tablet: real 0.008s user 0.005s sys 0.004s
I20260806 16:21:43.867247 9142 ts_tablet_manager.cc:1428] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 16:21:43.867687 8703 heartbeater.cc:503] Master 127.11.245.190:45285 was elected leader, sending a full tablet report...
I20260806 16:21:43.868175 8917 heartbeater.cc:503] Master 127.11.245.190:45285 was elected leader, sending a full tablet report...
I20260806 16:21:43.869630 9141 raft_consensus.cc:348] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86abc4ab6aa5458ab58bf64e0a8240a5" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 32801 } } peers { permanent_uuid: "5ab9b281b34e4ea48c732bdd766282a0" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 46531 } } peers { permanent_uuid: "91d70ac561d54da1ababa2a04b7582be" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45077 } }
I20260806 16:21:43.869958 9141 raft_consensus.cc:374] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:43.870023 9141 raft_consensus.cc:729] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab9b281b34e4ea48c732bdd766282a0, State: Initialized, Role: FOLLOWER
I20260806 16:21:43.870560 9141 consensus_queue.cc:260] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86abc4ab6aa5458ab58bf64e0a8240a5" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 32801 } } peers { permanent_uuid: "5ab9b281b34e4ea48c732bdd766282a0" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 46531 } } peers { permanent_uuid: "91d70ac561d54da1ababa2a04b7582be" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45077 } }
I20260806 16:21:43.871533 9110 heartbeater.cc:503] Master 127.11.245.190:45285 was elected leader, sending a full tablet report...
I20260806 16:21:43.872115 9141 ts_tablet_manager.cc:1428] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 16:21:43.965137 9149 raft_consensus.cc:482] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:21:43.965516 9149 raft_consensus.cc:504] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86abc4ab6aa5458ab58bf64e0a8240a5" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 32801 } } peers { permanent_uuid: "5ab9b281b34e4ea48c732bdd766282a0" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 46531 } } peers { permanent_uuid: "91d70ac561d54da1ababa2a04b7582be" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45077 } }
I20260806 16:21:43.967351 9149 leader_election.cc:288] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86abc4ab6aa5458ab58bf64e0a8240a5 (127.11.245.130:32801), 5ab9b281b34e4ea48c732bdd766282a0 (127.11.245.131:46531)
I20260806 16:21:43.970750 9148 raft_consensus.cc:482] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:21:43.971063 9148 raft_consensus.cc:504] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86abc4ab6aa5458ab58bf64e0a8240a5" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 32801 } } peers { permanent_uuid: "5ab9b281b34e4ea48c732bdd766282a0" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 46531 } } peers { permanent_uuid: "91d70ac561d54da1ababa2a04b7582be" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45077 } }
I20260806 16:21:43.972363 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.967602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:35001 (local address 127.11.245.130:32801)
0806 16:21:43.967854 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:43.967860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:43.968231 (+ 371us) server_negotiation.cc:407] Connection header received
0806 16:21:43.968469 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:43.968472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:43.968525 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:43.968623 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:43.968996 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.969667 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.971198 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.971689 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.971803 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.971854 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:43.972123 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 16:21:43.972217 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260806 16:21:43.972400 9152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.967528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.130:32801 (local address 127.11.245.129:35001)
0806 16:21:43.968091 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:43.968110 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:43.968325 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:43.968639 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:43.968642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:43.968661 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:43.968837 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.968843 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.969786 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.969790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.970999 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.971012 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.971825 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.971829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.971932 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.971965 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 16:21:43.972093 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 16:21:43.972191 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":52,"spinlock_wait_cycles":2816,"thread_start_us":172,"threads_started":1}
I20260806 16:21:43.972966 9148 leader_election.cc:288] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab9b281b34e4ea48c732bdd766282a0 (127.11.245.131:46531), 91d70ac561d54da1ababa2a04b7582be (127.11.245.129:45077)
I20260806 16:21:43.973381 9133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.968011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.129:60037 (local address 127.11.245.131:46531)
0806 16:21:43.968272 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:43.968276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:43.968293 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:21:43.968469 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:43.968472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:43.968548 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:43.968650 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:43.969071 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.969712 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.971897 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.972567 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.972686 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.972748 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:43.973124 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 16:21:43.973230 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 16:21:43.973676 8845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57570d979f8b4322bab140cd10bec521" candidate_uuid: "91d70ac561d54da1ababa2a04b7582be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86abc4ab6aa5458ab58bf64e0a8240a5" is_pre_election: true
I20260806 16:21:43.973815 9154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.967621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:46531 (local address 127.11.245.129:60037)
0806 16:21:43.968107 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:43.968122 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:43.968303 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:43.968675 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:43.968679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:43.968696 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:43.968904 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.968927 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.969861 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.969864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.971002 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.971014 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.972807 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.972810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.972946 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.972972 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 16:21:43.973556 (+ 584us) client_negotiation.cc:240] Negotiation successful
0806 16:21:43.973655 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":74,"spinlock_wait_cycles":35456,"thread_start_us":153,"threads_started":1}
I20260806 16:21:43.974001 8845 raft_consensus.cc:2454] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91d70ac561d54da1ababa2a04b7582be in term 0.
I20260806 16:21:43.974668 8523 leader_election.cc:302] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86abc4ab6aa5458ab58bf64e0a8240a5, 91d70ac561d54da1ababa2a04b7582be; no voters:
I20260806 16:21:43.974817 9047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57570d979f8b4322bab140cd10bec521" candidate_uuid: "91d70ac561d54da1ababa2a04b7582be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab9b281b34e4ea48c732bdd766282a0" is_pre_election: true
I20260806 16:21:43.975183 9047 raft_consensus.cc:2454] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91d70ac561d54da1ababa2a04b7582be in term 0.
I20260806 16:21:43.975270 9149 raft_consensus.cc:2781] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:21:43.975363 9149 raft_consensus.cc:482] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:21:43.975412 9149 raft_consensus.cc:3037] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:43.977546 9149 raft_consensus.cc:504] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86abc4ab6aa5458ab58bf64e0a8240a5" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 32801 } } peers { permanent_uuid: "5ab9b281b34e4ea48c732bdd766282a0" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 46531 } } peers { permanent_uuid: "91d70ac561d54da1ababa2a04b7582be" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45077 } }
I20260806 16:21:43.978492 9047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57570d979f8b4322bab140cd10bec521" candidate_uuid: "91d70ac561d54da1ababa2a04b7582be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab9b281b34e4ea48c732bdd766282a0"
I20260806 16:21:43.978674 9047 raft_consensus.cc:3037] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:43.978808 8921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.973174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.131:46531 (local address 127.11.245.130:56009)
0806 16:21:43.973361 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:43.973378 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:43.973477 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:43.974728 (+ 1251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:43.974731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:43.974747 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:43.974882 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.974889 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.976084 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.976088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.977174 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.977185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.978290 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.978293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.978408 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.978442 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 16:21:43.978516 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 16:21:43.978636 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":1}
I20260806 16:21:43.978943 9135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.973669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:58207 (local address 127.11.245.129:45077)
0806 16:21:43.973799 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:43.973803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:43.974555 (+ 752us) server_negotiation.cc:407] Connection header received
0806 16:21:43.974744 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:43.974747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:43.974795 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:43.974848 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:43.975280 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.975988 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.977636 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.978217 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.978328 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.978379 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:43.978730 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 16:21:43.978819 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 16:21:43.978971 9156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.973194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:45077 (local address 127.11.245.130:58207)
0806 16:21:43.974458 (+ 1264us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:43.974473 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:21:43.974625 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:21:43.974993 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:21:43.974997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:21:43.975018 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:21:43.975153 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.975160 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.976196 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.976205 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.977493 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:21:43.977506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.978440 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.978444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:21:43.978556 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.978582 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 16:21:43.978705 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 16:21:43.978781 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1196,"mutex_wait_us":62,"thread_start_us":110,"threads_started":1}
I20260806 16:21:43.979097 8845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57570d979f8b4322bab140cd10bec521" candidate_uuid: "91d70ac561d54da1ababa2a04b7582be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86abc4ab6aa5458ab58bf64e0a8240a5"
I20260806 16:21:43.979264 8845 raft_consensus.cc:3037] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:43.980266 9047 raft_consensus.cc:2454] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91d70ac561d54da1ababa2a04b7582be in term 1.
I20260806 16:21:43.980556 9149 leader_election.cc:288] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [CANDIDATE]: Term 1 election: Requested vote from peers 86abc4ab6aa5458ab58bf64e0a8240a5 (127.11.245.130:32801), 5ab9b281b34e4ea48c732bdd766282a0 (127.11.245.131:46531)
I20260806 16:21:43.980697 8845 raft_consensus.cc:2454] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91d70ac561d54da1ababa2a04b7582be in term 1.
I20260806 16:21:43.980719 8519 leader_election.cc:302] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab9b281b34e4ea48c732bdd766282a0, 91d70ac561d54da1ababa2a04b7582be; no voters:
I20260806 16:21:43.980903 9149 raft_consensus.cc:2781] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:43.981215 9149 raft_consensus.cc:686] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [term 1 LEADER]: Becoming Leader. State: Replica: 91d70ac561d54da1ababa2a04b7582be, State: Running, Role: LEADER
I20260806 16:21:43.981463 9133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:43.974234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.245.130:56009 (local address 127.11.245.131:46531)
0806 16:21:43.974378 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:43.974382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:43.974397 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:21:43.974456 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:43.974459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:43.974530 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:43.974608 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:21:43.975042 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.975840 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.977312 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:43.978025 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:43.978151 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:43.978217 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:43.981216 (+ 2999us) server_negotiation.cc:299] Negotiation successful
0806 16:21:43.981331 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 16:21:43.981573 8635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57570d979f8b4322bab140cd10bec521" candidate_uuid: "86abc4ab6aa5458ab58bf64e0a8240a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91d70ac561d54da1ababa2a04b7582be" is_pre_election: true
I20260806 16:21:43.981781 9149 consensus_queue.cc:237] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86abc4ab6aa5458ab58bf64e0a8240a5" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 32801 } } peers { permanent_uuid: "5ab9b281b34e4ea48c732bdd766282a0" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 46531 } } peers { permanent_uuid: "91d70ac561d54da1ababa2a04b7582be" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45077 } }
I20260806 16:21:43.981856 9047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57570d979f8b4322bab140cd10bec521" candidate_uuid: "86abc4ab6aa5458ab58bf64e0a8240a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab9b281b34e4ea48c732bdd766282a0" is_pre_election: true
I20260806 16:21:43.982056 9047 raft_consensus.cc:2379] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 86abc4ab6aa5458ab58bf64e0a8240a5 in current term 1: Already voted for candidate 91d70ac561d54da1ababa2a04b7582be in this term.
I20260806 16:21:43.982635 8741 leader_election.cc:302] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86abc4ab6aa5458ab58bf64e0a8240a5; no voters: 5ab9b281b34e4ea48c732bdd766282a0, 91d70ac561d54da1ababa2a04b7582be
I20260806 16:21:43.983008 9148 raft_consensus.cc:2726] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 16:21:43.986706 8402 catalog_manager.cc:5495] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be reported cstate change: term changed from 0 to 1, leader changed from <none> to 91d70ac561d54da1ababa2a04b7582be (127.11.245.129). New cstate: current_term: 1 leader_uuid: "91d70ac561d54da1ababa2a04b7582be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86abc4ab6aa5458ab58bf64e0a8240a5" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ab9b281b34e4ea48c732bdd766282a0" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 46531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91d70ac561d54da1ababa2a04b7582be" member_type: VOTER last_known_addr { host: "127.11.245.129" port: 45077 } health_report { overall_health: HEALTHY } } }
W20260806 16:21:44.014158 8919 tablet.cc:2369] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:21:44.015763 8704 tablet.cc:2369] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:21:44.024221 9111 tablet.cc:2369] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:21:44.032017 9135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:44.027489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41412 (local address 127.11.245.129:45077)
0806 16:21:44.027665 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:44.027671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:44.027693 (+ 22us) server_negotiation.cc:407] Connection header received
0806 16:21:44.027831 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:44.027833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:44.027930 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:44.028102 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:21:44.028396 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:44.028956 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:44.029899 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:44.030144 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:44.030244 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:44.030716 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:21:44.031446 (+ 730us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:21:44.031571 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:44.031786 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 16:21:44.031858 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 16:21:44.041592 8845 raft_consensus.cc:1261] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5 [term 1 FOLLOWER]: Refusing update from remote peer 91d70ac561d54da1ababa2a04b7582be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:21:44.041533 9047 raft_consensus.cc:1261] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0 [term 1 FOLLOWER]: Refusing update from remote peer 91d70ac561d54da1ababa2a04b7582be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:21:44.042594 9149 consensus_queue.cc:1035] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [LEADER]: Connected to new peer: Peer: permanent_uuid: "86abc4ab6aa5458ab58bf64e0a8240a5" member_type: VOTER last_known_addr { host: "127.11.245.130" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:21:44.043035 9157 consensus_queue.cc:1035] T 57570d979f8b4322bab140cd10bec521 P 91d70ac561d54da1ababa2a04b7582be [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab9b281b34e4ea48c732bdd766282a0" member_type: VOTER last_known_addr { host: "127.11.245.131" port: 46531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:21:44.051905 9163 mvcc.cc:205] Tried to move back new op lower bound from 7315592413342699520 to 7315592413114695680. Current Snapshot: MvccSnapshot[applied={T|T < 7315592413342699520}]
I20260806 16:21:44.053450 9169 mvcc.cc:205] Tried to move back new op lower bound from 7315592413342699520 to 7315592413114695680. Current Snapshot: MvccSnapshot[applied={T|T < 7315592413342699520}]
I20260806 16:21:44.055142 9167 mvcc.cc:205] Tried to move back new op lower bound from 7315592413342699520 to 7315592413114695680. Current Snapshot: MvccSnapshot[applied={T|T < 7315592413342699520}]
I20260806 16:21:44.064590 8402 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:21:44.064859 8402 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:21:44.068205 8402 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f13c69a545c041b08b63fa04912a4ac5: Sending DeleteTablet for 3 replicas of tablet 57570d979f8b4322bab140cd10bec521
I20260806 16:21:44.069538 8600 tablet_service.cc:1514] Processing DeleteTablet for tablet 57570d979f8b4322bab140cd10bec521 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:44 UTC) from {username='slave'} at 127.0.0.1:41400
I20260806 16:21:44.069581 8815 tablet_service.cc:1514] Processing DeleteTablet for tablet 57570d979f8b4322bab140cd10bec521 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:44 UTC) from {username='slave'} at 127.0.0.1:52612
I20260806 16:21:44.069945 9021 tablet_service.cc:1514] Processing DeleteTablet for tablet 57570d979f8b4322bab140cd10bec521 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:21:44 UTC) from {username='slave'} at 127.0.0.1:52290
I20260806 16:21:44.070433 9178 tablet_replica.cc:330] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5: stopping tablet replica
I20260806 16:21:44.070564 9182 tablet_replica.cc:330] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0: stopping tablet replica
I20260806 16:21:44.070928 9182 raft_consensus.cc:2229] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:21:44.070936 9178 raft_consensus.cc:2229] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:21:44.071084 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 8494
I20260806 16:21:44.071377 9178 raft_consensus.cc:2258] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:21:44.075135 9182 raft_consensus.cc:2258] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:21:44.077203 9178 ts_tablet_manager.cc:1905] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:21:44.080642 9182 ts_tablet_manager.cc:1905] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:21:44.081883 9178 ts_tablet_manager.cc:1918] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:21:44.082006 9178 log.cc:1196] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-1/wal/wals/57570d979f8b4322bab140cd10bec521
I20260806 16:21:44.082337 9178 ts_tablet_manager.cc:1939] T 57570d979f8b4322bab140cd10bec521 P 86abc4ab6aa5458ab58bf64e0a8240a5: Deleting consensus metadata
I20260806 16:21:44.084151 8383 catalog_manager.cc:4841] TS 86abc4ab6aa5458ab58bf64e0a8240a5 (127.11.245.130:32801): tablet 57570d979f8b4322bab140cd10bec521 (table test-table [id=63f23f9089ad4c25b526f592e119e252]) successfully deleted
I20260806 16:21:44.084618 9182 ts_tablet_manager.cc:1918] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:21:44.084729 9182 log.cc:1196] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0: Deleting WAL directory at /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786033011181791-12246-0/minicluster-data/ts-2/wal/wals/57570d979f8b4322bab140cd10bec521
I20260806 16:21:44.085042 9182 ts_tablet_manager.cc:1939] T 57570d979f8b4322bab140cd10bec521 P 5ab9b281b34e4ea48c732bdd766282a0: Deleting consensus metadata
I20260806 16:21:44.088727 8381 catalog_manager.cc:4841] TS 5ab9b281b34e4ea48c732bdd766282a0 (127.11.245.131:46531): tablet 57570d979f8b4322bab140cd10bec521 (table test-table [id=63f23f9089ad4c25b526f592e119e252]) successfully deleted
W20260806 16:21:44.096232 8382 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.245.129:45077 (error 108)
I20260806 16:21:44.097100 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 8706
I20260806 16:21:44.101828 9128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:44.101299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.129:45077 (local address 127.0.0.1:41428)
0806 16:21:44.101585 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 16:21:44.101690 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.129:45077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
W20260806 16:21:44.102070 8382 catalog_manager.cc:4584] TS 91d70ac561d54da1ababa2a04b7582be (127.11.245.129:45077): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 57570d979f8b4322bab140cd10bec521: Network error: Client connection negotiation failed: client connection to 127.11.245.129:45077: connect: Connection refused (error 111)
I20260806 16:21:44.116549 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 8923
I20260806 16:21:44.133605 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 8361
2026-08-06T16:21:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1478 ms)
[----------] 6 tests from EncryptionPolicyTest (13679 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:21:44 dist-test-slave-t3q3 krb5kdc[9188](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:21:44 dist-test-slave-t3q3 krb5kdc[9188](info): set up 2 sockets
Aug 06 16:21:44 dist-test-slave-t3q3 krb5kdc[9188](info): commencing operation
krb5kdc: starting...
W20260806 16:21:45.475634 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.000s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:21:45 dist-test-slave-t3q3 krb5kdc[9188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:21:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:21:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:21:45.639894 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33177
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:45.785426 9210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:45.785856 9210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:45.785948 9210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:45.790270 9210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:21:45.790400 9210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:45.790450 9210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:45.790506 9210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:21:45.790544 9210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:21:45.798368 9210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33177
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:45.800028 9210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:45.802289 9210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:45.813346 9218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:45.813346 9223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:45.813369 9217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:45.813412 9210 server_base.cc:1034] running on GCE node
I20260806 16:21:45.814483 9210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:45.815922 9210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:45.817106 9210 hybrid_clock.cc:648] HybridClock initialized: now 1786033305817085 us; error 40 us; skew 500 ppm
Aug 06 16:21:45 dist-test-slave-t3q3 krb5kdc[9188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033305, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:45.828800 9210 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:21:45.830266 9210 webserver.cc:460] Webserver started at http://127.0.0.1:37635/ using document root <none> and password file <none>
I20260806 16:21:45.831048 9210 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:45.831122 9210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:45.831372 9210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:45.834300 9210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "7ea2027db59c44649da0174d66f48519"
format_stamp: "Formatted at 2026-08-06 16:21:45 on dist-test-slave-t3q3"
server_key: "cafed945736fd3351429c907e34bccf0"
server_key_iv: "ff7860acfaea0d60d01b66641239d245"
server_key_version: "encryptionkey@0"
I20260806 16:21:45.835108 9210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "7ea2027db59c44649da0174d66f48519"
format_stamp: "Formatted at 2026-08-06 16:21:45 on dist-test-slave-t3q3"
server_key: "cafed945736fd3351429c907e34bccf0"
server_key_iv: "ff7860acfaea0d60d01b66641239d245"
server_key_version: "encryptionkey@0"
I20260806 16:21:45.840965 9210 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 16:21:45.844440 9233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:45.846472 9210 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:21:45.846705 9210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "7ea2027db59c44649da0174d66f48519"
format_stamp: "Formatted at 2026-08-06 16:21:45 on dist-test-slave-t3q3"
server_key: "cafed945736fd3351429c907e34bccf0"
server_key_iv: "ff7860acfaea0d60d01b66641239d245"
server_key_version: "encryptionkey@0"
I20260806 16:21:45.846885 9210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:45.861411 9210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:45.866080 9210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:45.866364 9210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:45.875856 9210 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33177
I20260806 16:21:45.875931 9308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33177 every 8 connection(s)
I20260806 16:21:45.877238 9210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:21:45.880978 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 9210
I20260806 16:21:45.881135 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:21:45.881760 12246 external_mini_cluster.cc:1428] Setting key e0d4f36f5945f91f3e03e32dc961e6da
I20260806 16:21:45.882076 9310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:45.890959 9310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519: Bootstrap starting.
Aug 06 16:21:45 dist-test-slave-t3q3 krb5kdc[9188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 16:21:45.895409 9310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:45.896944 9310 log.cc:824] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:45.900931 9310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519: No bootstrap required, opened a new log
I20260806 16:21:45.902642 9316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:45.883624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45207 (local address 127.0.0.1:33177)
0806 16:21:45.884330 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:45.884352 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:45.884404 (+ 52us) server_negotiation.cc:407] Connection header received
0806 16:21:45.885266 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:45.885291 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:45.885771 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:45.886487 (+ 716us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:45.887962 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:45.889190 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:45.890259 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:45.890857 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:45.891059 (+ 202us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:45.895217 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:45.895230 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:45.895261 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:45.895322 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:45.897955 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:45.898892 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:45.898901 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:45.898909 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:45.899110 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:45.899548 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:45.899551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:45.899553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:45.900261 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:45.900523 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:45.900993 (+ 470us) server_negotiation.cc:299] Negotiation successful
0806 16:21:45.901336 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":204,"threads_started":1}
I20260806 16:21:45.906563 9310 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea2027db59c44649da0174d66f48519" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33177 } }
I20260806 16:21:45.906987 9310 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:45.907101 9310 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ea2027db59c44649da0174d66f48519, State: Initialized, Role: FOLLOWER
I20260806 16:21:45.907775 9310 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea2027db59c44649da0174d66f48519" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33177 } }
I20260806 16:21:45.908013 9310 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:21:45.908169 9310 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:21:45.908325 9310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:45.908388 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:21:45.910138 9310 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea2027db59c44649da0174d66f48519" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33177 } }
I20260806 16:21:45.910789 9310 leader_election.cc:302] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ea2027db59c44649da0174d66f48519; no voters:
I20260806 16:21:45.911453 9310 leader_election.cc:288] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:21:45.911947 9322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:45.912425 9322 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [term 1 LEADER]: Becoming Leader. State: Replica: 7ea2027db59c44649da0174d66f48519, State: Running, Role: LEADER
I20260806 16:21:45.912998 9322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea2027db59c44649da0174d66f48519" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33177 } }
I20260806 16:21:45.913765 9310 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:21:45.915208 9322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ea2027db59c44649da0174d66f48519. Latest consensus state: current_term: 1 leader_uuid: "7ea2027db59c44649da0174d66f48519" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea2027db59c44649da0174d66f48519" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33177 } } }
I20260806 16:21:45.915534 9322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:45.915920 9328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ea2027db59c44649da0174d66f48519" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea2027db59c44649da0174d66f48519" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33177 } } }
I20260806 16:21:45.916102 9328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519 [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:45.916235 9345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:21:45.922104 9345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:21:45.924834 9316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:45.912001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34237 (local address 127.0.0.1:33177)
0806 16:21:45.912233 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:45.912239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:45.912692 (+ 453us) server_negotiation.cc:407] Connection header received
0806 16:21:45.912940 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:45.912944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:45.913011 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:45.913136 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:45.914291 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:45.915117 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:45.916421 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:45.916763 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:45.916845 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:45.920146 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:45.920150 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:45.920154 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:45.920191 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:45.922108 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:45.922887 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:45.922891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:45.922893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:45.922998 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:45.923483 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:45.923487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:45.923488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:45.923772 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:45.923911 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:45.924483 (+ 572us) server_negotiation.cc:299] Negotiation successful
0806 16:21:45.924623 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260806 16:21:45.931049 9345 catalog_manager.cc:1347] Generated new cluster ID: 3c274b61f5e343fb86d571884e7b68bc
I20260806 16:21:45.931169 9345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:21:45.945730 9345 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:21:45.947093 9345 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:21:45.955528 9345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ea2027db59c44649da0174d66f48519: Generated new TSK 0
I20260806 16:21:45.956667 9345 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:21:45.970501 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 9210
W20260806 16:21:45.987983 9184 connection.cc:535] client connection to 127.0.0.1:33177 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33177): Connection reset by peer (error 104)
2026-08-06T16:21:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1835 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:21:46 dist-test-slave-t3q3 krb5kdc[9363](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:21:46 dist-test-slave-t3q3 krb5kdc[9363](info): set up 2 sockets
Aug 06 16:21:46 dist-test-slave-t3q3 krb5kdc[9363](info): commencing operation
krb5kdc: starting...
W20260806 16:21:47.282996 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:21:47 dist-test-slave-t3q3 krb5kdc[9363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:21:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:21:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:21:47.455996 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:47.606942 9382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:47.607360 9382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:47.607450 9382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:47.612462 9382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:21:47.612578 9382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:47.612601 9382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:47.612620 9382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:21:47.612637 9382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:21:47.619539 9382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40201
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:47.621395 9382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:47.622967 9382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:47.634616 9393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:47.634605 9395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:47.634593 9392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:47.634994 9382 server_base.cc:1034] running on GCE node
I20260806 16:21:47.635936 9382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:47.638633 9382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:47.639915 9382 hybrid_clock.cc:648] HybridClock initialized: now 1786033307639901 us; error 42 us; skew 500 ppm
Aug 06 16:21:47 dist-test-slave-t3q3 krb5kdc[9363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033307, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:47.643993 9382 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:21:47.645403 9382 webserver.cc:460] Webserver started at http://127.0.0.1:41081/ using document root <none> and password file <none>
I20260806 16:21:47.646179 9382 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:47.646313 9382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:47.646608 9382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:47.649751 9382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "0d20271d42574868aaed827b85b6a745"
format_stamp: "Formatted at 2026-08-06 16:21:47 on dist-test-slave-t3q3"
server_key: "ab23a7d64dfc85170b0f1cbb44703687"
server_key_iv: "c6f35786a8325420c19440984be780a9"
server_key_version: "encryptionkey@0"
I20260806 16:21:47.650583 9382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "0d20271d42574868aaed827b85b6a745"
format_stamp: "Formatted at 2026-08-06 16:21:47 on dist-test-slave-t3q3"
server_key: "ab23a7d64dfc85170b0f1cbb44703687"
server_key_iv: "c6f35786a8325420c19440984be780a9"
server_key_version: "encryptionkey@0"
I20260806 16:21:47.656026 9382 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:21:47.659933 9406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:47.661706 9382 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260806 16:21:47.661891 9382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "0d20271d42574868aaed827b85b6a745"
format_stamp: "Formatted at 2026-08-06 16:21:47 on dist-test-slave-t3q3"
server_key: "ab23a7d64dfc85170b0f1cbb44703687"
server_key_iv: "c6f35786a8325420c19440984be780a9"
server_key_version: "encryptionkey@0"
I20260806 16:21:47.662055 9382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:47.688047 9382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:47.692873 9382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:47.693208 9382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:47.705097 9382 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40201
I20260806 16:21:47.705111 9483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40201 every 8 connection(s)
I20260806 16:21:47.706933 9382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:21:47.709265 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 9382
I20260806 16:21:47.709499 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:21:47.709908 12246 external_mini_cluster.cc:1428] Setting key 81098dfc67d6af3d212536916e5a1cad
I20260806 16:21:47.711685 9484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:47.719367 9484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745: Bootstrap starting.
I20260806 16:21:47.722874 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:47.711689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38987 (local address 127.0.0.1:40201)
0806 16:21:47.712364 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:47.712395 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:47.712442 (+ 47us) server_negotiation.cc:407] Connection header received
0806 16:21:47.713296 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:47.713317 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:47.713751 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:47.714182 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:47.714693 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:47.715810 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:47.717083 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:47.717644 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:47.717840 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:47.720241 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:47.720247 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:47.720264 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:47.720799 (+ 535us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:47.720873 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:47.720901 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:47.721032 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:47.721613 (+ 581us) server_negotiation.cc:299] Negotiation successful
0806 16:21:47.721859 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":215,"threads_started":1}
I20260806 16:21:47.723295 9484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:47.724372 9484 log.cc:824] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:47.727434 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:21:47.728003 9484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745: No bootstrap required, opened a new log
I20260806 16:21:47.733155 9484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d20271d42574868aaed827b85b6a745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } }
I20260806 16:21:47.733594 9484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:47.733670 9484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d20271d42574868aaed827b85b6a745, State: Initialized, Role: FOLLOWER
I20260806 16:21:47.734407 9484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d20271d42574868aaed827b85b6a745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } }
I20260806 16:21:47.734635 9484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:21:47.734776 9484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:21:47.734941 9484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:47.735112 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:47.730297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34769 (local address 127.0.0.1:40201)
0806 16:21:47.730440 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:47.730444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:47.730765 (+ 321us) server_negotiation.cc:407] Connection header received
0806 16:21:47.730952 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:47.730955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:47.731013 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:47.731103 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:47.731426 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:47.732003 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:47.732949 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:47.733212 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:47.733341 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:47.734496 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:47.734500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:47.734502 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:47.734517 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:47.734549 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:47.734582 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:47.734684 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:47.734857 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 16:21:47.734913 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 16:21:47.736721 9484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d20271d42574868aaed827b85b6a745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } }
I20260806 16:21:47.737475 9484 leader_election.cc:302] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d20271d42574868aaed827b85b6a745; no voters:
I20260806 16:21:47.737972 9484 leader_election.cc:288] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:21:47.738372 9496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:47.738876 9496 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [term 1 LEADER]: Becoming Leader. State: Replica: 0d20271d42574868aaed827b85b6a745, State: Running, Role: LEADER
I20260806 16:21:47.739303 9496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d20271d42574868aaed827b85b6a745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } }
I20260806 16:21:47.739907 9484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:21:47.741562 9499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d20271d42574868aaed827b85b6a745. Latest consensus state: current_term: 1 leader_uuid: "0d20271d42574868aaed827b85b6a745" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d20271d42574868aaed827b85b6a745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } } }
I20260806 16:21:47.741732 9499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:47.742129 9498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d20271d42574868aaed827b85b6a745" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d20271d42574868aaed827b85b6a745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } } }
I20260806 16:21:47.742236 9498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745 [sys.catalog]: This master's current role is: LEADER
W20260806 16:21:47.746140 9522 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:21:47.746246 9522 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:21:47.746346 9511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:21:47.747179 9511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:21:47.754748 9511 catalog_manager.cc:1347] Generated new cluster ID: 94e8ae2775574a97b77ff12bbb06806d
I20260806 16:21:47.754870 9511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:21:47.784668 9511 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:21:47.786464 9511 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:21:47.798108 9511 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d20271d42574868aaed827b85b6a745: Generated new TSK 0
I20260806 16:21:47.799333 9511 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:21:47.809618 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 9382
W20260806 16:21:47.824551 9361 connection.cc:535] client connection to 127.0.0.1:40201 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40201): Connection reset by peer (error 104)
2026-08-06T16:21:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1836 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:21:47 dist-test-slave-t3q3 krb5kdc[9535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:21:47 dist-test-slave-t3q3 krb5kdc[9535](info): set up 2 sockets
Aug 06 16:21:47 dist-test-slave-t3q3 krb5kdc[9535](info): commencing operation
krb5kdc: starting...
W20260806 16:21:49.154166 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.007s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:21:49 dist-test-slave-t3q3 krb5kdc[9535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:21:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:21:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:21:49.342773 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43217
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43217
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:49.501374 9561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:49.501771 9561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:49.501828 9561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:49.506356 9561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:21:49.506458 9561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:49.506480 9561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:49.506500 9561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:21:49.506515 9561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:21:49.512924 9561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43217
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:49.514633 9561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:49.516116 9561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:49.527873 9576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:49.527844 9579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:49.527943 9561 server_base.cc:1034] running on GCE node
W20260806 16:21:49.527884 9574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:49.529071 9561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:49.530589 9561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:49.533922 9561 hybrid_clock.cc:648] HybridClock initialized: now 1786033309533835 us; error 96 us; skew 500 ppm
Aug 06 16:21:49 dist-test-slave-t3q3 krb5kdc[9535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033309, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:49.538007 9561 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:21:49.539491 9561 webserver.cc:460] Webserver started at http://127.0.0.1:37219/ using document root <none> and password file <none>
I20260806 16:21:49.540449 9561 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:49.540567 9561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:49.540899 9561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:49.543654 9561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "a407b1744ddc49dc9e0521f9501813e8"
format_stamp: "Formatted at 2026-08-06 16:21:49 on dist-test-slave-t3q3"
server_key: "9d93d994a9141e3accc0c06bf8bb6243"
server_key_iv: "c12ab48fe6e6eb6f24ce3f5d89cd93a8"
server_key_version: "encryptionkey@0"
I20260806 16:21:49.544405 9561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "a407b1744ddc49dc9e0521f9501813e8"
format_stamp: "Formatted at 2026-08-06 16:21:49 on dist-test-slave-t3q3"
server_key: "9d93d994a9141e3accc0c06bf8bb6243"
server_key_iv: "c12ab48fe6e6eb6f24ce3f5d89cd93a8"
server_key_version: "encryptionkey@0"
I20260806 16:21:49.549618 9561 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 16:21:49.553109 9589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:49.554939 9561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:21:49.555147 9561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "a407b1744ddc49dc9e0521f9501813e8"
format_stamp: "Formatted at 2026-08-06 16:21:49 on dist-test-slave-t3q3"
server_key: "9d93d994a9141e3accc0c06bf8bb6243"
server_key_iv: "c12ab48fe6e6eb6f24ce3f5d89cd93a8"
server_key_version: "encryptionkey@0"
I20260806 16:21:49.555316 9561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:49.561410 9561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:49.566468 9561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:49.566745 9561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:49.576179 9561 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43217
I20260806 16:21:49.576196 9667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43217 every 8 connection(s)
I20260806 16:21:49.577467 9561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:21:49.581457 9668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:49.586184 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 9561
I20260806 16:21:49.586395 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:21:49.586772 12246 external_mini_cluster.cc:1428] Setting key b7b9f3be833e3410e6eaea41d2914869
I20260806 16:21:49.589509 9668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8: Bootstrap starting.
I20260806 16:21:49.592684 9668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:49.593595 9675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:49.588809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56197 (local address 127.0.0.1:43217)
0806 16:21:49.589370 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:49.589394 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:49.589439 (+ 45us) server_negotiation.cc:407] Connection header received
0806 16:21:49.590299 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:49.590321 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:49.590676 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:49.591102 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:49.592024 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:49.592030 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:49.592044 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:49.592054 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:49.592185 (+ 131us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:49.592212 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:49.592340 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:49.592470 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 16:21:49.592664 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":168,"threads_started":1}
I20260806 16:21:49.593688 9668 log.cc:824] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:49.596259 9668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8: No bootstrap required, opened a new log
I20260806 16:21:49.596939 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:21:49.601588 9675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:49.599412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47841 (local address 127.0.0.1:43217)
0806 16:21:49.599553 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:49.599557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:49.599795 (+ 238us) server_negotiation.cc:407] Connection header received
0806 16:21:49.599916 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:49.599921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:49.599979 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:49.600116 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:49.601048 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:49.601052 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:49.601055 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:49.601068 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:49.601099 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:49.601112 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:49.601210 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:49.601357 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 16:21:49.601413 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 16:21:49.601598 9668 raft_consensus.cc:348] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a407b1744ddc49dc9e0521f9501813e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43217 } }
I20260806 16:21:49.602049 9668 raft_consensus.cc:374] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:49.602116 9668 raft_consensus.cc:729] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a407b1744ddc49dc9e0521f9501813e8, State: Initialized, Role: FOLLOWER
I20260806 16:21:49.602648 9668 consensus_queue.cc:260] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a407b1744ddc49dc9e0521f9501813e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43217 } }
I20260806 16:21:49.602852 9668 raft_consensus.cc:388] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:21:49.602936 9668 raft_consensus.cc:482] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:21:49.603060 9668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:49.604336 9668 raft_consensus.cc:504] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a407b1744ddc49dc9e0521f9501813e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43217 } }
I20260806 16:21:49.604851 9668 leader_election.cc:302] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a407b1744ddc49dc9e0521f9501813e8; no voters:
I20260806 16:21:49.605302 9668 leader_election.cc:288] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:21:49.605664 9683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:49.606012 9683 raft_consensus.cc:686] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [term 1 LEADER]: Becoming Leader. State: Replica: a407b1744ddc49dc9e0521f9501813e8, State: Running, Role: LEADER
I20260806 16:21:49.606424 9683 consensus_queue.cc:237] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a407b1744ddc49dc9e0521f9501813e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43217 } }
I20260806 16:21:49.607203 9668 sys_catalog.cc:564] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:21:49.608628 9685 sys_catalog.cc:455] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a407b1744ddc49dc9e0521f9501813e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a407b1744ddc49dc9e0521f9501813e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43217 } } }
I20260806 16:21:49.608788 9685 sys_catalog.cc:458] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:49.609179 9686 sys_catalog.cc:455] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a407b1744ddc49dc9e0521f9501813e8. Latest consensus state: current_term: 1 leader_uuid: "a407b1744ddc49dc9e0521f9501813e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a407b1744ddc49dc9e0521f9501813e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43217 } } }
I20260806 16:21:49.609289 9686 sys_catalog.cc:458] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8 [sys.catalog]: This master's current role is: LEADER
W20260806 16:21:49.613595 9709 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:21:49.613723 9709 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:21:49.613816 9698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:21:49.614609 9698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:21:49.622702 9698 catalog_manager.cc:1347] Generated new cluster ID: 5fa8bf4f06fc4b26aae9ad8f2b6204f2
I20260806 16:21:49.622846 9698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:21:49.647509 9698 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:21:49.648152 9698 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:21:49.660291 9698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a407b1744ddc49dc9e0521f9501813e8: Generated new TSK 0
I20260806 16:21:49.661557 9698 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:21:49.677253 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 9561
2026-08-06T16:21:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1872 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:21:49 dist-test-slave-t3q3 krb5kdc[9718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:21:49 dist-test-slave-t3q3 krb5kdc[9718](info): set up 2 sockets
Aug 06 16:21:49 dist-test-slave-t3q3 krb5kdc[9718](info): commencing operation
krb5kdc: starting...
W20260806 16:21:51.013520 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.002s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:21:51 dist-test-slave-t3q3 krb5kdc[9718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:21:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:21:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:21:51.185526 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33909
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:51.330330 9734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:51.330688 9734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:51.330737 9734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:51.335458 9734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:21:51.335579 9734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:51.335602 9734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:51.335620 9734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:21:51.335636 9734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:21:51.342447 9734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:51.344148 9734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:51.345561 9734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:21:51.356752 9734 server_base.cc:1034] running on GCE node
W20260806 16:21:51.356609 9740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:51.356612 9743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:51.356688 9739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:51.357724 9734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:51.359185 9734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:51.360417 9734 hybrid_clock.cc:648] HybridClock initialized: now 1786033311360385 us; error 45 us; skew 500 ppm
Aug 06 16:21:51 dist-test-slave-t3q3 krb5kdc[9718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033311, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:51.368315 9734 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:21:51.369710 9734 webserver.cc:460] Webserver started at http://127.0.0.1:43911/ using document root <none> and password file <none>
I20260806 16:21:51.370532 9734 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:51.370643 9734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:51.370954 9734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:51.374228 9734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "69393674b58f4df8802ecf68a41ff7e1"
format_stamp: "Formatted at 2026-08-06 16:21:51 on dist-test-slave-t3q3"
server_key: "82fbb1f249d50e41a36281d4c3a0e4e4"
server_key_iv: "9111ae5004682298be6504a41bafd9ac"
server_key_version: "encryptionkey@0"
I20260806 16:21:51.375074 9734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "69393674b58f4df8802ecf68a41ff7e1"
format_stamp: "Formatted at 2026-08-06 16:21:51 on dist-test-slave-t3q3"
server_key: "82fbb1f249d50e41a36281d4c3a0e4e4"
server_key_iv: "9111ae5004682298be6504a41bafd9ac"
server_key_version: "encryptionkey@0"
I20260806 16:21:51.380968 9734 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:21:51.385499 9754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:51.387442 9734 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:21:51.387689 9734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "69393674b58f4df8802ecf68a41ff7e1"
format_stamp: "Formatted at 2026-08-06 16:21:51 on dist-test-slave-t3q3"
server_key: "82fbb1f249d50e41a36281d4c3a0e4e4"
server_key_iv: "9111ae5004682298be6504a41bafd9ac"
server_key_version: "encryptionkey@0"
I20260806 16:21:51.387882 9734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:51.404424 9734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:51.409476 9734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:51.409767 9734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:51.420481 9734 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33909
I20260806 16:21:51.420538 9841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33909 every 8 connection(s)
I20260806 16:21:51.421902 9734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:21:51.426230 9843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:51.427593 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 9734
I20260806 16:21:51.427800 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:21:51.428452 12246 external_mini_cluster.cc:1428] Setting key a8d19bd863ff246b8948abfee98acece
I20260806 16:21:51.436630 9843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1: Bootstrap starting.
I20260806 16:21:51.440814 9843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1: Neither blocks nor log segments found. Creating new log.
Aug 06 16:21:51 dist-test-slave-t3q3 krb5kdc[9718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 16:21:51.442052 9843 log.cc:824] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:51.445613 9843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1: No bootstrap required, opened a new log
I20260806 16:21:51.449842 9848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:51.430835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44835 (local address 127.0.0.1:33909)
0806 16:21:51.431515 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:51.431552 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:51.431773 (+ 221us) server_negotiation.cc:407] Connection header received
0806 16:21:51.433088 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:51.433119 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:51.433648 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:51.434124 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:51.435554 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:51.436898 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:51.438281 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:51.438871 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:51.439046 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:51.442856 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:51.442864 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:51.442886 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:51.442923 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:51.445579 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:51.446205 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:51.446212 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:51.446225 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:51.446398 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:51.446764 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:51.446767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:51.446770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:51.447426 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:51.447649 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:51.448036 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 16:21:51.448493 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":202,"threads_started":1}
I20260806 16:21:51.451200 9843 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69393674b58f4df8802ecf68a41ff7e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33909 } }
I20260806 16:21:51.451563 9843 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:51.451632 9843 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69393674b58f4df8802ecf68a41ff7e1, State: Initialized, Role: FOLLOWER
I20260806 16:21:51.452240 9843 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69393674b58f4df8802ecf68a41ff7e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33909 } }
I20260806 16:21:51.452454 9843 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:21:51.452534 9843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:21:51.452680 9843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:51.453876 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:21:51.454185 9843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69393674b58f4df8802ecf68a41ff7e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33909 } }
I20260806 16:21:51.454623 9843 leader_election.cc:302] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69393674b58f4df8802ecf68a41ff7e1; no voters:
I20260806 16:21:51.455071 9843 leader_election.cc:288] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:21:51.455173 9850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:51.455564 9850 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [term 1 LEADER]: Becoming Leader. State: Replica: 69393674b58f4df8802ecf68a41ff7e1, State: Running, Role: LEADER
I20260806 16:21:51.455973 9850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69393674b58f4df8802ecf68a41ff7e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33909 } }
I20260806 16:21:51.457052 9843 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:21:51.458134 9858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69393674b58f4df8802ecf68a41ff7e1. Latest consensus state: current_term: 1 leader_uuid: "69393674b58f4df8802ecf68a41ff7e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69393674b58f4df8802ecf68a41ff7e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33909 } } }
I20260806 16:21:51.458319 9858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:51.458715 9859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69393674b58f4df8802ecf68a41ff7e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69393674b58f4df8802ecf68a41ff7e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33909 } } }
I20260806 16:21:51.458827 9859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1 [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:51.459187 9871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:21:51.464653 9871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:21:51.467522 9848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:51.456591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33521 (local address 127.0.0.1:33909)
0806 16:21:51.456756 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:51.456761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:51.457066 (+ 305us) server_negotiation.cc:407] Connection header received
0806 16:21:51.457292 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:51.457295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:51.457374 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:51.457476 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:51.458711 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:51.459599 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:51.460924 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:51.461277 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:51.461424 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:51.463158 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:51.463162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:51.463166 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:51.463201 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:51.465261 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:51.465843 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:51.465846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:51.465848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:51.465943 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:51.466257 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:51.466261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:51.466263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:51.466519 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:51.466681 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:51.466974 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 16:21:51.467125 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":69}
I20260806 16:21:51.474524 9871 catalog_manager.cc:1347] Generated new cluster ID: d2c2d07da533489d94901b89b1af9951
I20260806 16:21:51.474750 9871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:21:51.488781 9871 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:21:51.490655 9871 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:21:51.504304 9871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69393674b58f4df8802ecf68a41ff7e1: Generated new TSK 0
I20260806 16:21:51.505681 9871 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:21:51.512115 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 9734
2026-08-06T16:21:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1835 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:21:51 dist-test-slave-t3q3 krb5kdc[9893](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:21:51 dist-test-slave-t3q3 krb5kdc[9893](info): set up 2 sockets
Aug 06 16:21:51 dist-test-slave-t3q3 krb5kdc[9893](info): commencing operation
krb5kdc: starting...
W20260806 16:21:52.854946 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.008s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:21:52 dist-test-slave-t3q3 krb5kdc[9893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:21:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:21:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:21:53.033674 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33009
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:53.188575 9916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:53.189002 9916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:53.189095 9916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:53.193563 9916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:21:53.193679 9916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:53.193719 9916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:53.193779 9916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:21:53.193818 9916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:21:53.199913 9916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:53.201550 9916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:53.202956 9916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:53.214018 9928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:53.214058 9924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:53.214085 9923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:53.214301 9916 server_base.cc:1034] running on GCE node
I20260806 16:21:53.215274 9916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:53.217101 9916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:53.220525 9916 hybrid_clock.cc:648] HybridClock initialized: now 1786033313220434 us; error 121 us; skew 500 ppm
Aug 06 16:21:53 dist-test-slave-t3q3 krb5kdc[9893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033313, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:53.224427 9916 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:21:53.225847 9916 webserver.cc:460] Webserver started at http://127.0.0.1:38521/ using document root <none> and password file <none>
I20260806 16:21:53.226702 9916 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:53.226815 9916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:53.227129 9916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:53.230118 9916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "5d88f7646a3540e080e6d390e87d848c"
format_stamp: "Formatted at 2026-08-06 16:21:53 on dist-test-slave-t3q3"
server_key: "9cfff73e2fa5384886f87cfc2a1cd6ff"
server_key_iv: "ff03ff87768ce752c9fe55ac04fb97fa"
server_key_version: "encryptionkey@0"
I20260806 16:21:53.230865 9916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "5d88f7646a3540e080e6d390e87d848c"
format_stamp: "Formatted at 2026-08-06 16:21:53 on dist-test-slave-t3q3"
server_key: "9cfff73e2fa5384886f87cfc2a1cd6ff"
server_key_iv: "ff03ff87768ce752c9fe55ac04fb97fa"
server_key_version: "encryptionkey@0"
I20260806 16:21:53.236260 9916 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:21:53.239966 9942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:53.241809 9916 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:21:53.242036 9916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "5d88f7646a3540e080e6d390e87d848c"
format_stamp: "Formatted at 2026-08-06 16:21:53 on dist-test-slave-t3q3"
server_key: "9cfff73e2fa5384886f87cfc2a1cd6ff"
server_key_iv: "ff03ff87768ce752c9fe55ac04fb97fa"
server_key_version: "encryptionkey@0"
I20260806 16:21:53.242218 9916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:53.259095 9916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:53.264309 9916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:53.264709 9916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:53.274536 9916 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33009
I20260806 16:21:53.274569 10013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33009 every 8 connection(s)
I20260806 16:21:53.276131 9916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:21:53.276418 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 9916
I20260806 16:21:53.276643 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:21:53.277076 12246 external_mini_cluster.cc:1428] Setting key b6d5dd14058f1262acd256d60036fcd5
I20260806 16:21:53.281018 10015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:53.288571 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:53.279085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37743 (local address 127.0.0.1:33009)
0806 16:21:53.279677 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:53.279696 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:53.279742 (+ 46us) server_negotiation.cc:407] Connection header received
0806 16:21:53.280645 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:53.280668 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:53.281152 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:53.281613 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:53.282119 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:53.283243 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:53.284319 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:53.284973 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:53.285172 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:53.286307 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:53.286313 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:53.286331 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:53.286848 (+ 517us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:53.286922 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:53.286950 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:53.287074 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:53.287248 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 16:21:53.287529 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":193,"threads_started":1}
I20260806 16:21:53.289050 10015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c: Bootstrap starting.
I20260806 16:21:53.292632 10015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:53.292910 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:21:53.293756 10015 log.cc:824] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:53.296933 10015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c: No bootstrap required, opened a new log
I20260806 16:21:53.300922 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:53.295542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46299 (local address 127.0.0.1:33009)
0806 16:21:53.295698 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:53.295703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:53.295897 (+ 194us) server_negotiation.cc:407] Connection header received
0806 16:21:53.296028 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:53.296031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:53.296117 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:53.296221 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:53.296556 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:53.297564 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:53.298577 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:53.298927 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:53.299054 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:53.300319 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:53.300334 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:53.300339 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:53.300362 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:53.300403 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:53.300420 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:53.300517 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:53.300673 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 16:21:53.300748 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 16:21:53.302954 10015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d88f7646a3540e080e6d390e87d848c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33009 } }
I20260806 16:21:53.303289 10015 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:53.303349 10015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d88f7646a3540e080e6d390e87d848c, State: Initialized, Role: FOLLOWER
I20260806 16:21:53.303864 10015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d88f7646a3540e080e6d390e87d848c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33009 } }
I20260806 16:21:53.304008 10015 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:21:53.304101 10015 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:21:53.304230 10015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:53.305455 10015 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d88f7646a3540e080e6d390e87d848c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33009 } }
I20260806 16:21:53.305881 10015 leader_election.cc:302] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d88f7646a3540e080e6d390e87d848c; no voters:
I20260806 16:21:53.306241 10015 leader_election.cc:288] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:21:53.306551 10032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:53.306898 10032 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [term 1 LEADER]: Becoming Leader. State: Replica: 5d88f7646a3540e080e6d390e87d848c, State: Running, Role: LEADER
I20260806 16:21:53.307302 10032 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d88f7646a3540e080e6d390e87d848c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33009 } }
I20260806 16:21:53.308158 10015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:21:53.309727 10033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d88f7646a3540e080e6d390e87d848c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d88f7646a3540e080e6d390e87d848c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33009 } } }
I20260806 16:21:53.309880 10033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:53.310273 10034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d88f7646a3540e080e6d390e87d848c. Latest consensus state: current_term: 1 leader_uuid: "5d88f7646a3540e080e6d390e87d848c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d88f7646a3540e080e6d390e87d848c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33009 } } }
I20260806 16:21:53.310386 10034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c [sys.catalog]: This master's current role is: LEADER
W20260806 16:21:53.314802 10058 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:21:53.314939 10058 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:21:53.315025 10045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:21:53.316020 10045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:21:53.324510 10045 catalog_manager.cc:1347] Generated new cluster ID: f50e54969d254523b78f96f583673e25
I20260806 16:21:53.324637 10045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:21:53.338951 10045 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:21:53.340381 10045 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:21:53.346807 10045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d88f7646a3540e080e6d390e87d848c: Generated new TSK 0
I20260806 16:21:53.347939 10045 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:21:53.365177 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 9916
2026-08-06T16:21:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1845 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:21:53 dist-test-slave-t3q3 krb5kdc[10069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:21:53 dist-test-slave-t3q3 krb5kdc[10069](info): set up 2 sockets
Aug 06 16:21:53 dist-test-slave-t3q3 krb5kdc[10069](info): commencing operation
krb5kdc: starting...
W20260806 16:21:54.696441 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:21:54 dist-test-slave-t3q3 krb5kdc[10069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:21:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:21:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:21:54.880865 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35129
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:55.038713 10085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:55.039143 10085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:55.039196 10085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:55.044121 10085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:21:55.044219 10085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:55.044240 10085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:55.044258 10085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:21:55.044275 10085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:21:55.050788 10085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:55.052573 10085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:55.054013 10085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:55.065145 10093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:55.065155 10092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:55.065155 10095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:55.065613 10085 server_base.cc:1034] running on GCE node
I20260806 16:21:55.066447 10085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:55.068037 10085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:55.069311 10085 hybrid_clock.cc:648] HybridClock initialized: now 1786033315069283 us; error 45 us; skew 500 ppm
Aug 06 16:21:55 dist-test-slave-t3q3 krb5kdc[10069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033315, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:55.073122 10085 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:21:55.074759 10085 webserver.cc:460] Webserver started at http://127.0.0.1:33379/ using document root <none> and password file <none>
I20260806 16:21:55.075706 10085 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:55.075848 10085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:55.076345 10085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:55.079844 10085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "4a36dcf61dbd49e692a92b60a014ea35"
format_stamp: "Formatted at 2026-08-06 16:21:55 on dist-test-slave-t3q3"
server_key: "c2a80463ce8ea0b84e1659e698c7ce84"
server_key_iv: "723bf8ff30b4eafe5a5303a7f9c2e243"
server_key_version: "encryptionkey@0"
I20260806 16:21:55.080765 10085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "4a36dcf61dbd49e692a92b60a014ea35"
format_stamp: "Formatted at 2026-08-06 16:21:55 on dist-test-slave-t3q3"
server_key: "c2a80463ce8ea0b84e1659e698c7ce84"
server_key_iv: "723bf8ff30b4eafe5a5303a7f9c2e243"
server_key_version: "encryptionkey@0"
I20260806 16:21:55.086201 10085 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:21:55.090154 10102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:55.092283 10085 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:21:55.092500 10085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "4a36dcf61dbd49e692a92b60a014ea35"
format_stamp: "Formatted at 2026-08-06 16:21:55 on dist-test-slave-t3q3"
server_key: "c2a80463ce8ea0b84e1659e698c7ce84"
server_key_iv: "723bf8ff30b4eafe5a5303a7f9c2e243"
server_key_version: "encryptionkey@0"
I20260806 16:21:55.092692 10085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:55.099206 10085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:55.104020 10085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:55.104350 10085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:55.114490 10085 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35129
I20260806 16:21:55.114529 10182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35129 every 8 connection(s)
I20260806 16:21:55.115888 10085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:21:55.121594 10186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:55.123842 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 10085
I20260806 16:21:55.124541 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:21:55.124956 12246 external_mini_cluster.cc:1428] Setting key e8822e49e4a48a92643c73ccb2ede4ae
I20260806 16:21:55.132906 10186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35: Bootstrap starting.
I20260806 16:21:55.133373 10189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:55.127862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58477 (local address 127.0.0.1:35129)
0806 16:21:55.128626 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:55.128647 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:55.128701 (+ 54us) server_negotiation.cc:407] Connection header received
0806 16:21:55.129569 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:55.129595 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:55.130021 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:55.130516 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:55.131619 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:55.131635 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:55.131655 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:55.131667 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:55.131737 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:55.131760 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:55.131964 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:55.132049 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 16:21:55.132316 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":213,"threads_started":1}
I20260806 16:21:55.136783 10186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:55.137665 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:21:55.138100 10186 log.cc:824] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:55.142532 10186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35: No bootstrap required, opened a new log
I20260806 16:21:55.143996 10189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:55.140687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60429 (local address 127.0.0.1:35129)
0806 16:21:55.140866 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:55.140872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:55.141339 (+ 467us) server_negotiation.cc:407] Connection header received
0806 16:21:55.141545 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:55.141549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:55.141595 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:55.141677 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:55.143229 (+ 1552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:55.143233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:55.143235 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:55.143258 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:55.143294 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:55.143306 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:55.143403 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:55.143752 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 16:21:55.143824 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 16:21:55.149972 10186 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a36dcf61dbd49e692a92b60a014ea35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35129 } }
I20260806 16:21:55.150375 10186 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:55.150417 10186 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a36dcf61dbd49e692a92b60a014ea35, State: Initialized, Role: FOLLOWER
I20260806 16:21:55.151012 10186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a36dcf61dbd49e692a92b60a014ea35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35129 } }
I20260806 16:21:55.151227 10186 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:21:55.151301 10186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:21:55.151439 10186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:55.153034 10186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a36dcf61dbd49e692a92b60a014ea35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35129 } }
I20260806 16:21:55.153586 10186 leader_election.cc:302] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a36dcf61dbd49e692a92b60a014ea35; no voters:
I20260806 16:21:55.154053 10186 leader_election.cc:288] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:21:55.154261 10197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:55.154613 10197 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [term 1 LEADER]: Becoming Leader. State: Replica: 4a36dcf61dbd49e692a92b60a014ea35, State: Running, Role: LEADER
I20260806 16:21:55.154999 10197 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a36dcf61dbd49e692a92b60a014ea35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35129 } }
I20260806 16:21:55.155877 10186 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:21:55.157267 10202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a36dcf61dbd49e692a92b60a014ea35. Latest consensus state: current_term: 1 leader_uuid: "4a36dcf61dbd49e692a92b60a014ea35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a36dcf61dbd49e692a92b60a014ea35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35129 } } }
I20260806 16:21:55.157261 10201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a36dcf61dbd49e692a92b60a014ea35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a36dcf61dbd49e692a92b60a014ea35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35129 } } }
I20260806 16:21:55.157485 10202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:55.157485 10201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35 [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:55.158104 10207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:21:55.162457 10207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:21:55.171782 10207 catalog_manager.cc:1347] Generated new cluster ID: b45bf76e8c2f4b92921e507391c7158d
I20260806 16:21:55.171928 10207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:21:55.191702 10207 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:21:55.192308 10207 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:21:55.200984 10207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a36dcf61dbd49e692a92b60a014ea35: Generated new TSK 0
I20260806 16:21:55.202458 10207 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:21:55.215063 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 10085
2026-08-06T16:21:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1857 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:21:55 dist-test-slave-t3q3 krb5kdc[10230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:21:55 dist-test-slave-t3q3 krb5kdc[10230](info): set up 2 sockets
Aug 06 16:21:55 dist-test-slave-t3q3 krb5kdc[10230](info): commencing operation
krb5kdc: starting...
W20260806 16:21:56.569342 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.000s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:21:56 dist-test-slave-t3q3 krb5kdc[10230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:21:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:21:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:21:56.753559 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35671
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:56.933985 10250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:56.934501 10250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:56.934595 10250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:56.947611 10250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:21:56.947806 10250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:56.947851 10250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:56.947896 10250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:21:56.947955 10250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:21:56.957522 10250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:56.959264 10250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:56.961586 10250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:56.975854 10259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:56.976019 10260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:56.975975 10250 server_base.cc:1034] running on GCE node
W20260806 16:21:56.976119 10264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:56.977504 10250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:56.979494 10250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:56.983037 10250 hybrid_clock.cc:648] HybridClock initialized: now 1786033316982977 us; error 164 us; skew 500 ppm
Aug 06 16:21:56 dist-test-slave-t3q3 krb5kdc[10230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033316, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:56.994668 10250 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:21:56.996717 10250 webserver.cc:460] Webserver started at http://127.0.0.1:43553/ using document root <none> and password file <none>
I20260806 16:21:56.997694 10250 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:56.997998 10250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:56.998378 10250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:57.001650 10250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "1fd331a64e404b1ca0cc6ca98dea813b"
format_stamp: "Formatted at 2026-08-06 16:21:56 on dist-test-slave-t3q3"
server_key: "820734b6e35801af31c7a52913cd26b7"
server_key_iv: "4b33fe2e74337a15bd536351311c5dfc"
server_key_version: "encryptionkey@0"
I20260806 16:21:57.002554 10250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "1fd331a64e404b1ca0cc6ca98dea813b"
format_stamp: "Formatted at 2026-08-06 16:21:56 on dist-test-slave-t3q3"
server_key: "820734b6e35801af31c7a52913cd26b7"
server_key_iv: "4b33fe2e74337a15bd536351311c5dfc"
server_key_version: "encryptionkey@0"
I20260806 16:21:57.008987 10250 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260806 16:21:57.013736 10275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:57.015930 10250 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:21:57.016220 10250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "1fd331a64e404b1ca0cc6ca98dea813b"
format_stamp: "Formatted at 2026-08-06 16:21:56 on dist-test-slave-t3q3"
server_key: "820734b6e35801af31c7a52913cd26b7"
server_key_iv: "4b33fe2e74337a15bd536351311c5dfc"
server_key_version: "encryptionkey@0"
I20260806 16:21:57.016429 10250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:57.037227 10250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:57.044957 10250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:57.045267 10250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:57.057560 10250 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35671
I20260806 16:21:57.057560 10372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35671 every 8 connection(s)
I20260806 16:21:57.059098 10250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:21:57.063783 10378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:57.068926 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 10250
I20260806 16:21:57.069125 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:21:57.069489 12246 external_mini_cluster.cc:1428] Setting key a82d1e9cc9722b851bed8f0339e70c9d
I20260806 16:21:57.074095 10378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b: Bootstrap starting.
I20260806 16:21:57.077816 10378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:57.079638 10378 log.cc:824] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:21:57 dist-test-slave-t3q3 krb5kdc[10230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 16:21:57.084566 10378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b: No bootstrap required, opened a new log
I20260806 16:21:57.092267 10378 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd331a64e404b1ca0cc6ca98dea813b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35671 } }
I20260806 16:21:57.092705 10378 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:57.092765 10378 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd331a64e404b1ca0cc6ca98dea813b, State: Initialized, Role: FOLLOWER
I20260806 16:21:57.092839 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:57.071475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:54353 (local address 127.0.0.1:35671)
0806 16:21:57.072239 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:57.072263 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:57.072326 (+ 63us) server_negotiation.cc:407] Connection header received
0806 16:21:57.073157 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:57.073183 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:57.073731 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:57.074211 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:57.075420 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:57.076717 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:57.078012 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:57.078774 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:57.079038 (+ 264us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:57.083613 (+ 4575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:57.083624 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:57.083651 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:57.083700 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:57.087810 (+ 4110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:57.088780 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:57.088791 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:57.088802 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:57.089039 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:57.089515 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:57.089521 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:57.089524 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:57.090309 (+ 785us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:57.090591 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:57.091022 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 16:21:57.091375 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":192,"threads_started":1}
I20260806 16:21:57.093437 10378 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd331a64e404b1ca0cc6ca98dea813b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35671 } }
I20260806 16:21:57.093611 10378 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:21:57.093710 10378 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:21:57.093858 10378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:57.095717 10378 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd331a64e404b1ca0cc6ca98dea813b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35671 } }
I20260806 16:21:57.096436 10378 leader_election.cc:302] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fd331a64e404b1ca0cc6ca98dea813b; no voters:
I20260806 16:21:57.097041 10378 leader_election.cc:288] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:21:57.097148 10386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:57.097488 10386 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [term 1 LEADER]: Becoming Leader. State: Replica: 1fd331a64e404b1ca0cc6ca98dea813b, State: Running, Role: LEADER
I20260806 16:21:57.098073 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:21:57.097970 10386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd331a64e404b1ca0cc6ca98dea813b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35671 } }
I20260806 16:21:57.099593 10378 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:21:57.100284 10388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fd331a64e404b1ca0cc6ca98dea813b. Latest consensus state: current_term: 1 leader_uuid: "1fd331a64e404b1ca0cc6ca98dea813b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd331a64e404b1ca0cc6ca98dea813b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35671 } } }
I20260806 16:21:57.100453 10388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:57.100857 10387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fd331a64e404b1ca0cc6ca98dea813b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd331a64e404b1ca0cc6ca98dea813b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35671 } } }
I20260806 16:21:57.100973 10387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:57.102473 10400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:21:57.107167 10400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:21:57.113847 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:57.101869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:56211 (local address 127.0.0.1:35671)
0806 16:21:57.102004 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:57.102011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:57.102755 (+ 744us) server_negotiation.cc:407] Connection header received
0806 16:21:57.102984 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:57.102991 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:57.103062 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:57.103182 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:57.104546 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:57.105368 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:57.106748 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:57.107103 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:57.107233 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:57.109542 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:57.109547 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:57.109550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:21:57.109594 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:57.111347 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:57.111981 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:57.111985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:57.111987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:57.112156 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:21:57.112545 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:21:57.112549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:21:57.112551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:21:57.112813 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:57.112985 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:57.113480 (+ 495us) server_negotiation.cc:299] Negotiation successful
0806 16:21:57.113622 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 16:21:57.116165 10400 catalog_manager.cc:1347] Generated new cluster ID: 702590a1974b4c12acfe7e46fc6ecdac
I20260806 16:21:57.116335 10400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:21:57.139885 10400 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:21:57.142745 10400 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:21:57.153527 10400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fd331a64e404b1ca0cc6ca98dea813b: Generated new TSK 0
I20260806 16:21:57.154982 10400 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:21:57.168447 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 10250
W20260806 16:21:57.187032 10228 connection.cc:535] client connection to 127.0.0.1:35671 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35671): Connection reset by peer (error 104)
2026-08-06T16:21:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1949 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:21:57 dist-test-slave-t3q3 krb5kdc[10424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:21:57 dist-test-slave-t3q3 krb5kdc[10424](info): set up 2 sockets
Aug 06 16:21:57 dist-test-slave-t3q3 krb5kdc[10424](info): commencing operation
krb5kdc: starting...
W20260806 16:21:58.502533 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:21:58 dist-test-slave-t3q3 krb5kdc[10424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:21:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:21:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:21:58.688696 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33483
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:21:58.862635 10444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:21:58.863168 10444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:21:58.863263 10444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:21:58.868163 10444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:21:58.868312 10444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:21:58.868352 10444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:21:58.868410 10444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:21:58.868466 10444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:21:58.875290 10444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:21:58.877103 10444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:21:58.878620 10444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:21:58.890403 10457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:58.890393 10453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:21:58.890439 10451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:21:58.891106 10444 server_base.cc:1034] running on GCE node
I20260806 16:21:58.892019 10444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:21:58.893814 10444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:21:58.895030 10444 hybrid_clock.cc:648] HybridClock initialized: now 1786033318895011 us; error 42 us; skew 500 ppm
Aug 06 16:21:58 dist-test-slave-t3q3 krb5kdc[10424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033318, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:21:58.899029 10444 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:21:58.900610 10444 webserver.cc:460] Webserver started at http://127.0.0.1:40809/ using document root <none> and password file <none>
I20260806 16:21:58.901438 10444 fs_manager.cc:359] Metadata directory not provided
I20260806 16:21:58.901551 10444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:21:58.901867 10444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:21:58.905119 10444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "7aefb49177cd492994fd569ab45ad147"
format_stamp: "Formatted at 2026-08-06 16:21:58 on dist-test-slave-t3q3"
server_key: "1d33a2c55da37ded0a2c85a698c3bacc"
server_key_iv: "55b4744f51cb3015c2eedc76de39042f"
server_key_version: "encryptionkey@0"
I20260806 16:21:58.905941 10444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "7aefb49177cd492994fd569ab45ad147"
format_stamp: "Formatted at 2026-08-06 16:21:58 on dist-test-slave-t3q3"
server_key: "1d33a2c55da37ded0a2c85a698c3bacc"
server_key_iv: "55b4744f51cb3015c2eedc76de39042f"
server_key_version: "encryptionkey@0"
I20260806 16:21:58.912405 10444 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:21:58.917042 10465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:21:58.919641 10444 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:21:58.920210 10444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "7aefb49177cd492994fd569ab45ad147"
format_stamp: "Formatted at 2026-08-06 16:21:58 on dist-test-slave-t3q3"
server_key: "1d33a2c55da37ded0a2c85a698c3bacc"
server_key_iv: "55b4744f51cb3015c2eedc76de39042f"
server_key_version: "encryptionkey@0"
I20260806 16:21:58.920434 10444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:21:58.944758 10444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:21:58.949175 10444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:21:58.949465 10444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:21:58.959131 10444 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33483
I20260806 16:21:58.959151 10547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33483 every 8 connection(s)
I20260806 16:21:58.960608 10444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:21:58.963204 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 10444
I20260806 16:21:58.963528 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:21:58.964371 12246 external_mini_cluster.cc:1428] Setting key 371988ef778957c72006af8cb2e990e6
I20260806 16:21:58.967922 10548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:21:58.975921 10548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147: Bootstrap starting.
I20260806 16:21:58.977387 10551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:58.967141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:36335 (local address 127.0.0.1:33483)
0806 16:21:58.967900 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:58.967922 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:58.967975 (+ 53us) server_negotiation.cc:407] Connection header received
0806 16:21:58.968930 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:58.968955 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:58.969435 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:58.969896 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:58.970551 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:58.971688 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:58.972799 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:58.973375 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:58.973569 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:58.975103 (+ 1534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:58.975110 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:58.975128 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:58.975645 (+ 517us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:58.975722 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:58.975750 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:58.975943 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:58.976206 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 16:21:58.976409 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":243,"threads_started":1}
I20260806 16:21:58.979948 10548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147: Neither blocks nor log segments found. Creating new log.
I20260806 16:21:58.981209 10548 log.cc:824] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147: Log is configured to *not* fsync() on all Append() calls
I20260806 16:21:58.981904 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:21:58.984627 10548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147: No bootstrap required, opened a new log
I20260806 16:21:58.989959 10551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:21:58.984800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:60441 (local address 127.0.0.1:33483)
0806 16:21:58.984952 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 16:21:58.984958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:21:58.985126 (+ 168us) server_negotiation.cc:407] Connection header received
0806 16:21:58.985244 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:21:58.985248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:21:58.985309 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:21:58.985412 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:21:58.985749 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:58.986532 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:58.987473 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:21:58.987831 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:21:58.987956 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:21:58.989310 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:21:58.989315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:21:58.989318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:21:58.989335 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:21:58.989371 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:21:58.989388 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:21:58.989489 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 16:21:58.989710 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 16:21:58.989780 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:21:58.990658 10548 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aefb49177cd492994fd569ab45ad147" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33483 } }
I20260806 16:21:58.991052 10548 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:21:58.991113 10548 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aefb49177cd492994fd569ab45ad147, State: Initialized, Role: FOLLOWER
I20260806 16:21:58.991676 10548 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aefb49177cd492994fd569ab45ad147" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33483 } }
I20260806 16:21:58.991833 10548 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:21:58.991884 10548 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:21:58.991977 10548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:21:58.993264 10548 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aefb49177cd492994fd569ab45ad147" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33483 } }
I20260806 16:21:58.993703 10548 leader_election.cc:302] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7aefb49177cd492994fd569ab45ad147; no voters:
I20260806 16:21:58.994064 10548 leader_election.cc:288] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:21:58.994230 10562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:21:58.994529 10562 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [term 1 LEADER]: Becoming Leader. State: Replica: 7aefb49177cd492994fd569ab45ad147, State: Running, Role: LEADER
I20260806 16:21:58.994921 10562 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aefb49177cd492994fd569ab45ad147" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33483 } }
I20260806 16:21:58.995756 10548 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:21:58.996984 10567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7aefb49177cd492994fd569ab45ad147. Latest consensus state: current_term: 1 leader_uuid: "7aefb49177cd492994fd569ab45ad147" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aefb49177cd492994fd569ab45ad147" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33483 } } }
I20260806 16:21:58.997145 10567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:58.997633 10575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:21:58.997650 10566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7aefb49177cd492994fd569ab45ad147" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aefb49177cd492994fd569ab45ad147" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33483 } } }
I20260806 16:21:58.997865 10566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147 [sys.catalog]: This master's current role is: LEADER
I20260806 16:21:59.002543 10575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:21:59.013199 10575 catalog_manager.cc:1347] Generated new cluster ID: 2b1232bdd9cd4e309394dcea5c00a3d3
I20260806 16:21:59.013352 10575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:21:59.038148 10575 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:21:59.039541 10575 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:21:59.047120 10575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7aefb49177cd492994fd569ab45ad147: Generated new TSK 0
I20260806 16:21:59.048246 10575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:21:59.056526 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 10444
W20260806 16:21:59.078452 10422 connection.cc:535] client connection to 127.0.0.1:33483 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33483): Connection reset by peer (error 104)
2026-08-06T16:21:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1891 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:21:59 dist-test-slave-t3q3 krb5kdc[10593](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:21:59 dist-test-slave-t3q3 krb5kdc[10593](info): set up 2 sockets
Aug 06 16:21:59 dist-test-slave-t3q3 krb5kdc[10593](info): commencing operation
krb5kdc: starting...
W20260806 16:22:00.446082 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.351s user 0.001s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:22:00 dist-test-slave-t3q3 krb5kdc[10593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:22:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:22:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:22:00.625185 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40017
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:22:00.791851 10614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:22:00.792382 10614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:22:00.792438 10614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:22:00.798435 10614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:22:00.798583 10614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:22:00.798605 10614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:22:00.798627 10614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:22:00.798643 10614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:22:00.805855 10614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:22:00.807657 10614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:22:00.809228 10614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:22:00.821580 10622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:00.821573 10619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:00.821571 10620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:00.822526 10614 server_base.cc:1034] running on GCE node
I20260806 16:22:00.823927 10614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:22:00.825789 10614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:22:00.827080 10614 hybrid_clock.cc:648] HybridClock initialized: now 1786033320827045 us; error 58 us; skew 500 ppm
Aug 06 16:22:00 dist-test-slave-t3q3 krb5kdc[10593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033320, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:22:00.831324 10614 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:22:00.833437 10614 webserver.cc:460] Webserver started at http://127.0.0.1:45919/ using document root <none> and password file <none>
I20260806 16:22:00.834401 10614 fs_manager.cc:359] Metadata directory not provided
I20260806 16:22:00.834491 10614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:22:00.834762 10614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:22:00.839787 10614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "beb5c30ee4f542308b8677495e4c837a"
format_stamp: "Formatted at 2026-08-06 16:22:00 on dist-test-slave-t3q3"
server_key: "76662f740138c4f028893ad21b16f576"
server_key_iv: "431cb05b43afeeb9802ac79aa2387a38"
server_key_version: "encryptionkey@0"
I20260806 16:22:00.840983 10614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "beb5c30ee4f542308b8677495e4c837a"
format_stamp: "Formatted at 2026-08-06 16:22:00 on dist-test-slave-t3q3"
server_key: "76662f740138c4f028893ad21b16f576"
server_key_iv: "431cb05b43afeeb9802ac79aa2387a38"
server_key_version: "encryptionkey@0"
I20260806 16:22:00.847503 10614 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 16:22:00.852659 10630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:22:00.855234 10614 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:22:00.855468 10614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "beb5c30ee4f542308b8677495e4c837a"
format_stamp: "Formatted at 2026-08-06 16:22:00 on dist-test-slave-t3q3"
server_key: "76662f740138c4f028893ad21b16f576"
server_key_iv: "431cb05b43afeeb9802ac79aa2387a38"
server_key_version: "encryptionkey@0"
I20260806 16:22:00.855675 10614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:22:00.865556 10614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:22:00.870720 10614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:22:00.870985 10614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:22:00.886628 10614 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40017
I20260806 16:22:00.886770 10708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40017 every 8 connection(s)
I20260806 16:22:00.888474 10614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:22:00.888770 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 10614
I20260806 16:22:00.888980 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:22:00.889488 12246 external_mini_cluster.cc:1428] Setting key 5c4c055e2b12eeda02a310f8313cdf5c
I20260806 16:22:00.894011 10711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:22:00.901934 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:00.891918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:36807 (local address 127.0.0.1:40017)
0806 16:22:00.892886 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:00.892908 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:00.892960 (+ 52us) server_negotiation.cc:407] Connection header received
0806 16:22:00.893963 (+ 1003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:00.893989 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:00.894394 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:00.894804 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:00.896027 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:00.896034 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:00.896053 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:22:00.896148 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:00.896236 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:22:00.896302 (+ 66us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:00.896489 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:00.896765 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 16:22:00.896977 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":249,"threads_started":1}
I20260806 16:22:00.904793 10711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a: Bootstrap starting.
I20260806 16:22:00.909008 10711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a: Neither blocks nor log segments found. Creating new log.
I20260806 16:22:00.909310 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:22:00.910445 10711 log.cc:824] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a: Log is configured to *not* fsync() on all Append() calls
I20260806 16:22:00.915177 10711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a: No bootstrap required, opened a new log
I20260806 16:22:00.915686 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:00.912644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:60997 (local address 127.0.0.1:40017)
0806 16:22:00.912801 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:00.912806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:00.913317 (+ 511us) server_negotiation.cc:407] Connection header received
0806 16:22:00.913584 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:00.913587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:00.913633 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:00.913722 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:00.915093 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:00.915104 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:00.915107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:22:00.915120 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:00.915155 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:22:00.915168 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:00.915268 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:00.915435 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 16:22:00.915503 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 16:22:00.920841 10711 raft_consensus.cc:348] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb5c30ee4f542308b8677495e4c837a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40017 } }
I20260806 16:22:00.921240 10711 raft_consensus.cc:374] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:22:00.921311 10711 raft_consensus.cc:729] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb5c30ee4f542308b8677495e4c837a, State: Initialized, Role: FOLLOWER
I20260806 16:22:00.921963 10711 consensus_queue.cc:260] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb5c30ee4f542308b8677495e4c837a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40017 } }
I20260806 16:22:00.922175 10711 raft_consensus.cc:388] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:22:00.922276 10711 raft_consensus.cc:482] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:22:00.922415 10711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:22:00.923763 10711 raft_consensus.cc:504] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb5c30ee4f542308b8677495e4c837a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40017 } }
I20260806 16:22:00.924335 10711 leader_election.cc:302] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beb5c30ee4f542308b8677495e4c837a; no voters:
I20260806 16:22:00.925056 10711 leader_election.cc:288] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:22:00.925400 10729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:22:00.925937 10729 raft_consensus.cc:686] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [term 1 LEADER]: Becoming Leader. State: Replica: beb5c30ee4f542308b8677495e4c837a, State: Running, Role: LEADER
I20260806 16:22:00.926386 10729 consensus_queue.cc:237] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb5c30ee4f542308b8677495e4c837a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40017 } }
I20260806 16:22:00.927088 10711 sys_catalog.cc:564] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:22:00.928753 10732 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [sys.catalog]: SysCatalogTable state changed. Reason: New leader beb5c30ee4f542308b8677495e4c837a. Latest consensus state: current_term: 1 leader_uuid: "beb5c30ee4f542308b8677495e4c837a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb5c30ee4f542308b8677495e4c837a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40017 } } }
I20260806 16:22:00.928938 10732 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:00.929383 10731 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "beb5c30ee4f542308b8677495e4c837a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb5c30ee4f542308b8677495e4c837a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40017 } } }
I20260806 16:22:00.929571 10731 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a [sys.catalog]: This master's current role is: LEADER
W20260806 16:22:00.933678 10752 catalog_manager.cc:1558] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:22:00.933820 10752 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:22:00.933933 10744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:22:00.934803 10744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:22:00.943989 10744 catalog_manager.cc:1347] Generated new cluster ID: 87bb92058b1b454c829b2b59e6b9eee3
I20260806 16:22:00.944217 10744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:22:00.970942 10744 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:22:00.971470 10744 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:22:00.980937 10744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P beb5c30ee4f542308b8677495e4c837a: Generated new TSK 0
I20260806 16:22:00.982110 10744 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:22:00.994617 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 10614
2026-08-06T16:22:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1941 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:22:01 dist-test-slave-t3q3 krb5kdc[10762](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:22:01 dist-test-slave-t3q3 krb5kdc[10762](info): set up 2 sockets
Aug 06 16:22:01 dist-test-slave-t3q3 krb5kdc[10762](info): commencing operation
krb5kdc: starting...
W20260806 16:22:02.354327 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.318s user 0.002s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:22:02 dist-test-slave-t3q3 krb5kdc[10762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:22:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:22:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:22:02.532871 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44899
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:22:02.711525 10782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:22:02.711954 10782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:22:02.712006 10782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:22:02.717075 10782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:22:02.717191 10782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:22:02.717214 10782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:22:02.717234 10782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:22:02.717252 10782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:22:02.724573 10782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44899
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:22:02.726446 10782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:22:02.727988 10782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:22:02.745285 10793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:02.745550 10798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:02.745550 10795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:02.745893 10782 server_base.cc:1034] running on GCE node
I20260806 16:22:02.747627 10782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:22:02.749410 10782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:22:02.750725 10782 hybrid_clock.cc:648] HybridClock initialized: now 1786033322750675 us; error 69 us; skew 500 ppm
Aug 06 16:22:02 dist-test-slave-t3q3 krb5kdc[10762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033322, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:22:02.761538 10782 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:22:02.763541 10782 webserver.cc:460] Webserver started at http://127.0.0.1:41843/ using document root <none> and password file <none>
I20260806 16:22:02.764503 10782 fs_manager.cc:359] Metadata directory not provided
I20260806 16:22:02.764598 10782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:22:02.764868 10782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:22:02.767832 10782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "21664a7fc0244c6fa339d65cd140d047"
format_stamp: "Formatted at 2026-08-06 16:22:02 on dist-test-slave-t3q3"
server_key: "bfe9d6cad1eea0f5a09366b3c5585fb5"
server_key_iv: "e484ea3631ca806e3ad4d3a8acddebde"
server_key_version: "encryptionkey@0"
I20260806 16:22:02.768720 10782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "21664a7fc0244c6fa339d65cd140d047"
format_stamp: "Formatted at 2026-08-06 16:22:02 on dist-test-slave-t3q3"
server_key: "bfe9d6cad1eea0f5a09366b3c5585fb5"
server_key_iv: "e484ea3631ca806e3ad4d3a8acddebde"
server_key_version: "encryptionkey@0"
I20260806 16:22:02.774223 10782 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:22:02.778534 10805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:22:02.781185 10782 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 16:22:02.781424 10782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "21664a7fc0244c6fa339d65cd140d047"
format_stamp: "Formatted at 2026-08-06 16:22:02 on dist-test-slave-t3q3"
server_key: "bfe9d6cad1eea0f5a09366b3c5585fb5"
server_key_iv: "e484ea3631ca806e3ad4d3a8acddebde"
server_key_version: "encryptionkey@0"
I20260806 16:22:02.781610 10782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:22:02.795909 10782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:22:02.801079 10782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:22:02.801417 10782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:22:02.812312 10782 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44899
I20260806 16:22:02.812338 10874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44899 every 8 connection(s)
I20260806 16:22:02.813925 10782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:22:02.818303 10876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:22:02.818581 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 10782
I20260806 16:22:02.818782 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:22:02.819159 12246 external_mini_cluster.cc:1428] Setting key 95c3fce0fbc48adf8ab94c99ef72759f
I20260806 16:22:02.829246 10876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047: Bootstrap starting.
Aug 06 16:22:02 dist-test-slave-t3q3 krb5kdc[10762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 16:22:02.832973 10876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047: Neither blocks nor log segments found. Creating new log.
I20260806 16:22:02.834029 10876 log.cc:824] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047: Log is configured to *not* fsync() on all Append() calls
I20260806 16:22:02.837334 10876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047: No bootstrap required, opened a new log
I20260806 16:22:02.839066 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:02.821302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:41153 (local address 127.0.0.1:44899)
0806 16:22:02.821898 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:02.821918 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:02.821963 (+ 45us) server_negotiation.cc:407] Connection header received
0806 16:22:02.823122 (+ 1159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:02.823152 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:02.823668 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:02.824196 (+ 528us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:02.825328 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:02.826527 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:02.827539 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:02.828315 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:02.828547 (+ 232us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:02.832146 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:02.832155 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:02.832176 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:02.832226 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:02.834555 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:02.835672 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:02.835683 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:02.835695 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:02.835883 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:02.836371 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:02.836374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:02.836376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:02.837043 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:02.837318 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:02.837733 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 16:22:02.838084 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":182,"threads_started":1}
I20260806 16:22:02.842935 10876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21664a7fc0244c6fa339d65cd140d047" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44899 } }
I20260806 16:22:02.843390 10876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:22:02.843475 10876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21664a7fc0244c6fa339d65cd140d047, State: Initialized, Role: FOLLOWER
I20260806 16:22:02.843559 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:22:02.844136 10876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21664a7fc0244c6fa339d65cd140d047" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44899 } }
I20260806 16:22:02.844309 10876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:22:02.844362 10876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:22:02.844501 10876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:22:02.845870 10876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21664a7fc0244c6fa339d65cd140d047" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44899 } }
I20260806 16:22:02.846324 10876 leader_election.cc:302] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21664a7fc0244c6fa339d65cd140d047; no voters:
I20260806 16:22:02.846753 10876 leader_election.cc:288] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:22:02.846839 10889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:22:02.847115 10889 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [term 1 LEADER]: Becoming Leader. State: Replica: 21664a7fc0244c6fa339d65cd140d047, State: Running, Role: LEADER
I20260806 16:22:02.847566 10889 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21664a7fc0244c6fa339d65cd140d047" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44899 } }
I20260806 16:22:02.848655 10876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:22:02.849198 10889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21664a7fc0244c6fa339d65cd140d047" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21664a7fc0244c6fa339d65cd140d047" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44899 } } }
I20260806 16:22:02.849397 10889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:02.849735 10890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21664a7fc0244c6fa339d65cd140d047. Latest consensus state: current_term: 1 leader_uuid: "21664a7fc0244c6fa339d65cd140d047" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21664a7fc0244c6fa339d65cd140d047" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44899 } } }
I20260806 16:22:02.849939 10890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047 [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:02.850523 10896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:22:02.855564 10896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:22:02.861724 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:02.847508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:55557 (local address 127.0.0.1:44899)
0806 16:22:02.847661 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:02.847685 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:02.850034 (+ 2349us) server_negotiation.cc:407] Connection header received
0806 16:22:02.850304 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:02.850308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:02.850380 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:02.850481 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:02.852504 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:02.853391 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:02.854297 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:02.854716 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:02.854819 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:02.856834 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:02.856840 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:02.856844 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:02.856880 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:02.858736 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:02.859368 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:02.859372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:02.859374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:02.859470 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:02.859892 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:02.859896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:02.859898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:02.860193 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:02.860344 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:02.861455 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0806 16:22:02.861586 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 16:22:02.864825 10896 catalog_manager.cc:1347] Generated new cluster ID: 6d2dd993dd4e497ea1293d6d021d090c
I20260806 16:22:02.864959 10896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:22:02.925604 10896 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:22:02.926986 10896 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:22:02.933759 10896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21664a7fc0244c6fa339d65cd140d047: Generated new TSK 0
I20260806 16:22:02.934782 10896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:22:02.950412 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 10782
W20260806 16:22:02.970038 10758 connection.cc:535] client connection to 127.0.0.1:44899 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44899): Connection reset by peer (error 104)
2026-08-06T16:22:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1950 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:22:03 dist-test-slave-t3q3 krb5kdc[10920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:22:03 dist-test-slave-t3q3 krb5kdc[10920](info): set up 2 sockets
Aug 06 16:22:03 dist-test-slave-t3q3 krb5kdc[10920](info): commencing operation
krb5kdc: starting...
W20260806 16:22:04.308745 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:22:04 dist-test-slave-t3q3 krb5kdc[10920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:22:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:22:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:22:04.485313 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41803
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:22:04.654627 10946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:22:04.655102 10946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:22:04.655202 10946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:22:04.661257 10946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:22:04.661410 10946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:22:04.661449 10946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:22:04.661480 10946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:22:04.661542 10946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:22:04.668874 10946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41803
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:22:04.670627 10946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:22:04.672279 10946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:22:04.684986 10958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:04.685111 10955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:04.685111 10956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:04.685747 10946 server_base.cc:1034] running on GCE node
I20260806 16:22:04.686690 10946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:22:04.688956 10946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:22:04.690276 10946 hybrid_clock.cc:648] HybridClock initialized: now 1786033324690220 us; error 64 us; skew 500 ppm
Aug 06 16:22:04 dist-test-slave-t3q3 krb5kdc[10920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033324, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:22:04.694795 10946 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:22:04.696290 10946 webserver.cc:460] Webserver started at http://127.0.0.1:37339/ using document root <none> and password file <none>
I20260806 16:22:04.697044 10946 fs_manager.cc:359] Metadata directory not provided
I20260806 16:22:04.697188 10946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:22:04.697449 10946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:22:04.700944 10946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "68f3aa93796d4aeba9da888b892c00fd"
format_stamp: "Formatted at 2026-08-06 16:22:04 on dist-test-slave-t3q3"
server_key: "fda9320d0a6ce5d873d80314a3714b37"
server_key_iv: "73ea5350d08a703a7876a496a5c6536e"
server_key_version: "encryptionkey@0"
I20260806 16:22:04.702206 10946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "68f3aa93796d4aeba9da888b892c00fd"
format_stamp: "Formatted at 2026-08-06 16:22:04 on dist-test-slave-t3q3"
server_key: "fda9320d0a6ce5d873d80314a3714b37"
server_key_iv: "73ea5350d08a703a7876a496a5c6536e"
server_key_version: "encryptionkey@0"
I20260806 16:22:04.708837 10946 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260806 16:22:04.712509 10967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:22:04.714468 10946 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:22:04.714696 10946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "68f3aa93796d4aeba9da888b892c00fd"
format_stamp: "Formatted at 2026-08-06 16:22:04 on dist-test-slave-t3q3"
server_key: "fda9320d0a6ce5d873d80314a3714b37"
server_key_iv: "73ea5350d08a703a7876a496a5c6536e"
server_key_version: "encryptionkey@0"
I20260806 16:22:04.714885 10946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:22:04.746105 10946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:22:04.751856 10946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:22:04.752224 10946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:22:04.763039 10946 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41803
I20260806 16:22:04.763073 11038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41803 every 8 connection(s)
I20260806 16:22:04.764981 10946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:22:04.769466 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 10946
I20260806 16:22:04.769677 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:22:04.770114 12246 external_mini_cluster.cc:1428] Setting key d78318272046cff259f2293e895b611d
I20260806 16:22:04.770459 11039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:22:04.781957 11039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd: Bootstrap starting.
I20260806 16:22:04.784385 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:04.772491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:56099 (local address 127.0.0.1:41803)
0806 16:22:04.773104 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:04.773125 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:04.773173 (+ 48us) server_negotiation.cc:407] Connection header received
0806 16:22:04.774562 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:04.774591 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:04.775098 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:04.775499 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:04.776028 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:04.777339 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:04.779048 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:04.779651 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:04.779814 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:04.781683 (+ 1869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:04.781690 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:04.781711 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:22:04.782347 (+ 636us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:04.782436 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:22:04.782468 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:04.782610 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:04.782853 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 16:22:04.783080 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":221,"threads_started":1}
I20260806 16:22:04.787704 11039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd: Neither blocks nor log segments found. Creating new log.
I20260806 16:22:04.788978 11039 log.cc:824] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd: Log is configured to *not* fsync() on all Append() calls
I20260806 16:22:04.790372 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:22:04.792573 11039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd: No bootstrap required, opened a new log
I20260806 16:22:04.799614 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:04.793833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:57655 (local address 127.0.0.1:41803)
0806 16:22:04.794114 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:04.794120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:04.794296 (+ 176us) server_negotiation.cc:407] Connection header received
0806 16:22:04.794442 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:04.794447 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:04.794508 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:04.794615 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:04.794968 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:04.795824 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:04.796929 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:04.797300 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:04.797428 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:04.798733 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:04.798737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:04.798740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:22:04.798761 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:04.798800 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:22:04.798815 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:04.798939 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:04.799378 (+ 439us) server_negotiation.cc:299] Negotiation successful
0806 16:22:04.799439 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 16:22:04.799217 11039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f3aa93796d4aeba9da888b892c00fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41803 } }
I20260806 16:22:04.799752 11039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:22:04.799819 11039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f3aa93796d4aeba9da888b892c00fd, State: Initialized, Role: FOLLOWER
I20260806 16:22:04.800617 11039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f3aa93796d4aeba9da888b892c00fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41803 } }
I20260806 16:22:04.800845 11039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:22:04.801069 11039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:22:04.801211 11039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:22:04.802574 11039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f3aa93796d4aeba9da888b892c00fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41803 } }
I20260806 16:22:04.803066 11039 leader_election.cc:302] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68f3aa93796d4aeba9da888b892c00fd; no voters:
I20260806 16:22:04.803560 11039 leader_election.cc:288] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:22:04.803752 11053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:22:04.804042 11053 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [term 1 LEADER]: Becoming Leader. State: Replica: 68f3aa93796d4aeba9da888b892c00fd, State: Running, Role: LEADER
I20260806 16:22:04.804525 11053 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f3aa93796d4aeba9da888b892c00fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41803 } }
I20260806 16:22:04.805315 11039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:22:04.806666 11054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68f3aa93796d4aeba9da888b892c00fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f3aa93796d4aeba9da888b892c00fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41803 } } }
I20260806 16:22:04.806855 11054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:04.807348 11065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:22:04.808367 11055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68f3aa93796d4aeba9da888b892c00fd. Latest consensus state: current_term: 1 leader_uuid: "68f3aa93796d4aeba9da888b892c00fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f3aa93796d4aeba9da888b892c00fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41803 } } }
I20260806 16:22:04.808707 11055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:04.811429 11065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:22:04.820415 11065 catalog_manager.cc:1347] Generated new cluster ID: 439f6b23fe3b43d2b45bff5135a444f8
I20260806 16:22:04.820565 11065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:22:04.834544 11065 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:22:04.836133 11065 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:22:04.844395 11065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68f3aa93796d4aeba9da888b892c00fd: Generated new TSK 0
I20260806 16:22:04.845818 11065 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:22:04.855194 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 10946
W20260806 16:22:04.870586 10915 connection.cc:535] client connection to 127.0.0.1:41803 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41803): Connection reset by peer (error 104)
2026-08-06T16:22:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1900 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:22:04 dist-test-slave-t3q3 krb5kdc[11087](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:22:04 dist-test-slave-t3q3 krb5kdc[11087](info): set up 2 sockets
Aug 06 16:22:04 dist-test-slave-t3q3 krb5kdc[11087](info): commencing operation
krb5kdc: starting...
W20260806 16:22:06.209374 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.001s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:22:06 dist-test-slave-t3q3 krb5kdc[11087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:22:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:22:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:22:06.409893 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33063
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:22:06.588514 11111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:22:06.588953 11111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:22:06.589010 11111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:22:06.594172 11111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:22:06.594293 11111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:22:06.594316 11111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:22:06.594334 11111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:22:06.594357 11111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:22:06.602031 11111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33063
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:22:06.603933 11111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:22:06.606389 11111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:22:06.618415 11123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:06.618417 11125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:06.619025 11111 server_base.cc:1034] running on GCE node
W20260806 16:22:06.618408 11121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:06.620216 11111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:22:06.621978 11111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:22:06.623226 11111 hybrid_clock.cc:648] HybridClock initialized: now 1786033326623215 us; error 48 us; skew 500 ppm
Aug 06 16:22:06 dist-test-slave-t3q3 krb5kdc[11087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033326, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:22:06.627034 11111 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:22:06.628496 11111 webserver.cc:460] Webserver started at http://127.0.0.1:37115/ using document root <none> and password file <none>
I20260806 16:22:06.629300 11111 fs_manager.cc:359] Metadata directory not provided
I20260806 16:22:06.629413 11111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:22:06.629720 11111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:22:06.632753 11111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "f50a7ff8d29d4b9d820adae9da562810"
format_stamp: "Formatted at 2026-08-06 16:22:06 on dist-test-slave-t3q3"
server_key: "c7878dc1b4df174edb4181044c4c2d92"
server_key_iv: "9ac82ad685b94ce5e70ce2c4e5a369b9"
server_key_version: "encryptionkey@0"
I20260806 16:22:06.633610 11111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "f50a7ff8d29d4b9d820adae9da562810"
format_stamp: "Formatted at 2026-08-06 16:22:06 on dist-test-slave-t3q3"
server_key: "c7878dc1b4df174edb4181044c4c2d92"
server_key_iv: "9ac82ad685b94ce5e70ce2c4e5a369b9"
server_key_version: "encryptionkey@0"
I20260806 16:22:06.639317 11111 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:22:06.643487 11134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:22:06.645911 11111 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:22:06.646229 11111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "f50a7ff8d29d4b9d820adae9da562810"
format_stamp: "Formatted at 2026-08-06 16:22:06 on dist-test-slave-t3q3"
server_key: "c7878dc1b4df174edb4181044c4c2d92"
server_key_iv: "9ac82ad685b94ce5e70ce2c4e5a369b9"
server_key_version: "encryptionkey@0"
I20260806 16:22:06.646450 11111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:22:06.654639 11111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:22:06.662479 11111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:22:06.662807 11111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:22:06.675222 11217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33063 every 8 connection(s)
I20260806 16:22:06.675262 11111 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33063
I20260806 16:22:06.676947 11111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:22:06.681294 11218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:22:06.684612 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 11111
I20260806 16:22:06.684827 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:22:06.685202 12246 external_mini_cluster.cc:1428] Setting key edada7eb9ef53d64f16bab2e666607b8
I20260806 16:22:06.690615 11218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810: Bootstrap starting.
I20260806 16:22:06.693138 11221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:06.687364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:47969 (local address 127.0.0.1:33063)
0806 16:22:06.688258 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:06.688281 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:06.688337 (+ 56us) server_negotiation.cc:407] Connection header received
0806 16:22:06.689406 (+ 1069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:06.689439 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:06.689864 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:06.690436 (+ 572us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:06.691468 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:06.691474 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:06.691490 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:22:06.691499 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:06.691554 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:22:06.691575 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:06.691762 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:06.691849 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 16:22:06.692118 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":212,"threads_started":1}
I20260806 16:22:06.694149 11218 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810: Neither blocks nor log segments found. Creating new log.
I20260806 16:22:06.695176 11218 log.cc:824] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810: Log is configured to *not* fsync() on all Append() calls
I20260806 16:22:06.697999 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:22:06.698900 11218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810: No bootstrap required, opened a new log
I20260806 16:22:06.703502 11221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:06.700965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:40927 (local address 127.0.0.1:33063)
0806 16:22:06.701130 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:06.701141 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:06.701365 (+ 224us) server_negotiation.cc:407] Connection header received
0806 16:22:06.701530 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:06.701533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:06.701586 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:06.701670 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:06.702842 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:06.702855 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:06.702857 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:22:06.702871 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:06.702905 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:22:06.702917 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:06.703010 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:06.703219 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 16:22:06.703311 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 16:22:06.705858 11218 raft_consensus.cc:348] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50a7ff8d29d4b9d820adae9da562810" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33063 } }
I20260806 16:22:06.706218 11218 raft_consensus.cc:374] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:22:06.706259 11218 raft_consensus.cc:729] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f50a7ff8d29d4b9d820adae9da562810, State: Initialized, Role: FOLLOWER
I20260806 16:22:06.707162 11218 consensus_queue.cc:260] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50a7ff8d29d4b9d820adae9da562810" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33063 } }
I20260806 16:22:06.707355 11218 raft_consensus.cc:388] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:22:06.707470 11218 raft_consensus.cc:482] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:22:06.707617 11218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:22:06.709056 11218 raft_consensus.cc:504] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50a7ff8d29d4b9d820adae9da562810" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33063 } }
I20260806 16:22:06.709569 11218 leader_election.cc:302] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f50a7ff8d29d4b9d820adae9da562810; no voters:
I20260806 16:22:06.710471 11218 leader_election.cc:288] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:22:06.710608 11232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:22:06.711160 11232 raft_consensus.cc:686] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [term 1 LEADER]: Becoming Leader. State: Replica: f50a7ff8d29d4b9d820adae9da562810, State: Running, Role: LEADER
I20260806 16:22:06.711778 11232 consensus_queue.cc:237] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50a7ff8d29d4b9d820adae9da562810" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33063 } }
I20260806 16:22:06.712591 11218 sys_catalog.cc:564] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:22:06.714574 11236 sys_catalog.cc:455] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f50a7ff8d29d4b9d820adae9da562810. Latest consensus state: current_term: 1 leader_uuid: "f50a7ff8d29d4b9d820adae9da562810" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50a7ff8d29d4b9d820adae9da562810" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33063 } } }
I20260806 16:22:06.714648 11234 sys_catalog.cc:455] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f50a7ff8d29d4b9d820adae9da562810" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50a7ff8d29d4b9d820adae9da562810" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33063 } } }
I20260806 16:22:06.714843 11236 sys_catalog.cc:458] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:06.714843 11234 sys_catalog.cc:458] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810 [sys.catalog]: This master's current role is: LEADER
W20260806 16:22:06.721148 11254 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:22:06.721568 11254 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:22:06.721686 11248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:22:06.723802 11248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:22:06.736557 11248 catalog_manager.cc:1347] Generated new cluster ID: 00df190b82eb40a8af67781454a3570c
I20260806 16:22:06.736765 11248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:22:06.765264 11248 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:22:06.765829 11248 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:22:06.772212 11248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f50a7ff8d29d4b9d820adae9da562810: Generated new TSK 0
I20260806 16:22:06.773262 11248 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:22:06.780229 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 11111
2026-08-06T16:22:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1924 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:22:06 dist-test-slave-t3q3 krb5kdc[11268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:22:06 dist-test-slave-t3q3 krb5kdc[11268](info): set up 2 sockets
Aug 06 16:22:06 dist-test-slave-t3q3 krb5kdc[11268](info): commencing operation
krb5kdc: starting...
W20260806 16:22:08.109515 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:22:08 dist-test-slave-t3q3 krb5kdc[11268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:22:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:22:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:22:08.293211 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:46057
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:38071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:46057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:22:08.453347 11287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:22:08.453733 11287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:22:08.453783 11287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:22:08.458385 11287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:22:08.458495 11287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:22:08.458518 11287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:22:08.458537 11287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:22:08.458554 11287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:22:08.465201 11287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:38071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:46057
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:46057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:22:08.466707 11287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:22:08.468053 11287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:22:08.480605 11297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:08.480580 11300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:08.480589 11298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:08.482697 11287 server_base.cc:1034] running on GCE node
I20260806 16:22:08.483677 11287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:22:08.485682 11287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:22:08.486992 11287 hybrid_clock.cc:648] HybridClock initialized: now 1786033328486954 us; error 64 us; skew 500 ppm
Aug 06 16:22:08 dist-test-slave-t3q3 krb5kdc[11268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033328, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:22:08.496572 11287 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:22:08.497987 11287 webserver.cc:460] Webserver started at http://127.11.245.190:42647/ using document root <none> and password file <none>
I20260806 16:22:08.498751 11287 fs_manager.cc:359] Metadata directory not provided
I20260806 16:22:08.498826 11287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:22:08.499063 11287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:22:08.502823 11287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "fbdbee292e304d1d93ee99e656788ef7"
format_stamp: "Formatted at 2026-08-06 16:22:08 on dist-test-slave-t3q3"
server_key: "26f27e49e9e2189a3a972dc53b8c7c03"
server_key_iv: "3a3c4a39c8f6ae9b7dfb03438f5de4b2"
server_key_version: "encryptionkey@0"
I20260806 16:22:08.503939 11287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "fbdbee292e304d1d93ee99e656788ef7"
format_stamp: "Formatted at 2026-08-06 16:22:08 on dist-test-slave-t3q3"
server_key: "26f27e49e9e2189a3a972dc53b8c7c03"
server_key_iv: "3a3c4a39c8f6ae9b7dfb03438f5de4b2"
server_key_version: "encryptionkey@0"
I20260806 16:22:08.510028 11287 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.001s
I20260806 16:22:08.514377 11312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:22:08.516433 11287 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:22:08.516649 11287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "fbdbee292e304d1d93ee99e656788ef7"
format_stamp: "Formatted at 2026-08-06 16:22:08 on dist-test-slave-t3q3"
server_key: "26f27e49e9e2189a3a972dc53b8c7c03"
server_key_iv: "3a3c4a39c8f6ae9b7dfb03438f5de4b2"
server_key_version: "encryptionkey@0"
I20260806 16:22:08.516834 11287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:22:08.532584 11287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:22:08.537365 11287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:22:08.537667 11287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:22:08.549160 11287 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:46057
I20260806 16:22:08.549170 11393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:46057 every 8 connection(s)
I20260806 16:22:08.550612 11287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:22:08.554952 11394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:22:08.556952 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 11287
I20260806 16:22:08.557168 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:22:08.557538 12246 external_mini_cluster.cc:1428] Setting key 0cd85463c3c832b010bd07ef11a65629
I20260806 16:22:08.563617 11394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7: Bootstrap starting.
I20260806 16:22:08.566843 11394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7: Neither blocks nor log segments found. Creating new log.
I20260806 16:22:08.567981 11394 log.cc:824] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:22:08 dist-test-slave-t3q3 krb5kdc[11268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:22:08.571055 11394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7: No bootstrap required, opened a new log
I20260806 16:22:08.576467 11394 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbdbee292e304d1d93ee99e656788ef7" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46057 } }
I20260806 16:22:08.576833 11394 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:22:08.576836 11397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:08.559784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42155 (local address 127.11.245.190:46057)
0806 16:22:08.560467 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:08.560503 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:08.560551 (+ 48us) server_negotiation.cc:407] Connection header received
0806 16:22:08.561389 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:08.561414 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:08.561874 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:08.562326 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:08.563354 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:08.564806 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:08.566101 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:08.566995 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:08.567242 (+ 247us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:08.570825 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:08.570832 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:08.570852 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:08.570887 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:08.572933 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:08.573616 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:08.573623 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:08.573630 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:08.573762 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:08.574163 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:08.574174 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:08.574176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:08.574704 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:08.575045 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:08.575405 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 16:22:08.575712 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":241,"threads_started":1}
I20260806 16:22:08.576938 11394 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbdbee292e304d1d93ee99e656788ef7, State: Initialized, Role: FOLLOWER
I20260806 16:22:08.577636 11394 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbdbee292e304d1d93ee99e656788ef7" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46057 } }
I20260806 16:22:08.577849 11394 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:22:08.577922 11394 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:22:08.578033 11394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:22:08.579615 11394 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbdbee292e304d1d93ee99e656788ef7" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46057 } }
I20260806 16:22:08.580149 11394 leader_election.cc:302] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbdbee292e304d1d93ee99e656788ef7; no voters:
I20260806 16:22:08.580691 11394 leader_election.cc:288] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:22:08.580821 11400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:22:08.581068 11400 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [term 1 LEADER]: Becoming Leader. State: Replica: fbdbee292e304d1d93ee99e656788ef7, State: Running, Role: LEADER
I20260806 16:22:08.581413 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:22:08.581569 11400 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbdbee292e304d1d93ee99e656788ef7" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46057 } }
I20260806 16:22:08.582510 11394 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:22:08.583832 11403 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbdbee292e304d1d93ee99e656788ef7. Latest consensus state: current_term: 1 leader_uuid: "fbdbee292e304d1d93ee99e656788ef7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbdbee292e304d1d93ee99e656788ef7" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46057 } } }
I20260806 16:22:08.584008 11403 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:08.584355 11402 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbdbee292e304d1d93ee99e656788ef7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbdbee292e304d1d93ee99e656788ef7" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 46057 } } }
I20260806 16:22:08.584466 11402 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7 [sys.catalog]: This master's current role is: LEADER
W20260806 16:22:08.589761 11433 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:22:08.589884 11433 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:22:08.589973 11419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:22:08.591167 11419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:22:08.597083 11397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:08.584417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39739 (local address 127.11.245.190:46057)
0806 16:22:08.584563 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:08.584569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:08.587114 (+ 2545us) server_negotiation.cc:407] Connection header received
0806 16:22:08.587827 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:08.587834 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:08.587911 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:08.588014 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:08.588926 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:08.589649 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:08.590664 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:08.590964 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:08.591097 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:08.592977 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:08.592981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:08.592987 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:08.593035 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:08.594797 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:08.595459 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:08.595463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:08.595465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:08.595565 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:08.595972 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:08.595975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:08.595977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:08.596274 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:08.596443 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:08.596777 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 16:22:08.596897 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:22:08.601195 11419 catalog_manager.cc:1347] Generated new cluster ID: 1c1d6d7af258424db293c990dd6c60b5
I20260806 16:22:08.601334 11419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:22:08.616266 11419 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:22:08.618131 11419 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:22:08.626255 11419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbdbee292e304d1d93ee99e656788ef7: Generated new TSK 0
I20260806 16:22:08.627367 11419 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:22:08.643569 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 11287
W20260806 16:22:08.675660 11265 connection.cc:535] client connection to 127.11.245.190:46057 recv error: Network error: failed to read from TLS socket (remote: 127.11.245.190:46057): Connection reset by peer (error 104)
2026-08-06T16:22:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1878 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:22:08 dist-test-slave-t3q3 krb5kdc[11441](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:22:08 dist-test-slave-t3q3 krb5kdc[11441](info): set up 2 sockets
Aug 06 16:22:08 dist-test-slave-t3q3 krb5kdc[11441](info): commencing operation
krb5kdc: starting...
W20260806 16:22:10.011036 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.000s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:22:10 dist-test-slave-t3q3 krb5kdc[11441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:22:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:22:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:22:10.208371 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:44701
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:45273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:44701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:22:10.366807 11459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:22:10.367276 11459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:22:10.367370 11459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:22:10.371840 11459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:22:10.371979 11459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:22:10.372020 11459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:22:10.372088 11459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:22:10.372121 11459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:22:10.378379 11459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:45273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:44701
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:44701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:22:10.380085 11459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:22:10.381520 11459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:22:10.392822 11468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:10.392767 11467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:10.392940 11459 server_base.cc:1034] running on GCE node
W20260806 16:22:10.392767 11470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:10.394004 11459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:22:10.395614 11459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:22:10.396845 11459 hybrid_clock.cc:648] HybridClock initialized: now 1786033330396800 us; error 73 us; skew 500 ppm
Aug 06 16:22:10 dist-test-slave-t3q3 krb5kdc[11441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:22:10.401331 11459 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:22:10.402710 11459 webserver.cc:460] Webserver started at http://127.11.245.190:40029/ using document root <none> and password file <none>
I20260806 16:22:10.403499 11459 fs_manager.cc:359] Metadata directory not provided
I20260806 16:22:10.403611 11459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:22:10.403921 11459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:22:10.408272 11459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "d58701b9c49549838e3d54d37586ef91"
format_stamp: "Formatted at 2026-08-06 16:22:10 on dist-test-slave-t3q3"
server_key: "149f8ca37dfe07e89fd7e6b4a64571f0"
server_key_iv: "1dad2091519d7f7a0d18cc09a3f1fa58"
server_key_version: "encryptionkey@0"
I20260806 16:22:10.409209 11459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "d58701b9c49549838e3d54d37586ef91"
format_stamp: "Formatted at 2026-08-06 16:22:10 on dist-test-slave-t3q3"
server_key: "149f8ca37dfe07e89fd7e6b4a64571f0"
server_key_iv: "1dad2091519d7f7a0d18cc09a3f1fa58"
server_key_version: "encryptionkey@0"
I20260806 16:22:10.415503 11459 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 16:22:10.419801 11478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:22:10.422237 11459 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:22:10.422489 11459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "d58701b9c49549838e3d54d37586ef91"
format_stamp: "Formatted at 2026-08-06 16:22:10 on dist-test-slave-t3q3"
server_key: "149f8ca37dfe07e89fd7e6b4a64571f0"
server_key_iv: "1dad2091519d7f7a0d18cc09a3f1fa58"
server_key_version: "encryptionkey@0"
I20260806 16:22:10.422685 11459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:22:10.439492 11459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:22:10.446012 11459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:22:10.446863 11459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:22:10.458966 11459 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:44701
I20260806 16:22:10.459416 11548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:44701 every 8 connection(s)
I20260806 16:22:10.460683 11459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:22:10.461941 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 11459
I20260806 16:22:10.462095 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:22:10.462527 12246 external_mini_cluster.cc:1428] Setting key 3eb5a68957d42dc2b5fdcc9e8c6f5bda
I20260806 16:22:10.466228 11549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:22:10.475818 11557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:10.464599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41377 (local address 127.11.245.190:44701)
0806 16:22:10.465198 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:10.465223 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:10.465281 (+ 58us) server_negotiation.cc:407] Connection header received
0806 16:22:10.466551 (+ 1270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:10.466582 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:10.467150 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:10.467612 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:10.468205 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:10.469818 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:10.470848 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:10.471603 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:10.471795 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:10.473070 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:10.473079 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:10.473102 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:22:10.473736 (+ 634us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:10.473832 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:22:10.473873 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:10.474062 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:10.474292 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 16:22:10.474480 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":179,"threads_started":1}
I20260806 16:22:10.475835 11549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91: Bootstrap starting.
I20260806 16:22:10.480511 11549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91: Neither blocks nor log segments found. Creating new log.
I20260806 16:22:10.480574 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:22:10.481545 11549 log.cc:824] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91: Log is configured to *not* fsync() on all Append() calls
I20260806 16:22:10.484684 11549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91: No bootstrap required, opened a new log
I20260806 16:22:10.488446 11557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:10.483320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34925 (local address 127.11.245.190:44701)
0806 16:22:10.483520 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:10.483525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:10.483685 (+ 160us) server_negotiation.cc:407] Connection header received
0806 16:22:10.483880 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:10.483888 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:10.483936 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:10.484034 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:10.484382 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:10.485026 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:10.485960 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:10.486208 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:10.486333 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:10.487797 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:10.487806 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:10.487809 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:22:10.487827 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:10.487859 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:22:10.487871 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:10.487987 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:10.488180 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 16:22:10.488233 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 16:22:10.491593 11549 raft_consensus.cc:348] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58701b9c49549838e3d54d37586ef91" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44701 } }
I20260806 16:22:10.491979 11549 raft_consensus.cc:374] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:22:10.492043 11549 raft_consensus.cc:729] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58701b9c49549838e3d54d37586ef91, State: Initialized, Role: FOLLOWER
I20260806 16:22:10.492755 11549 consensus_queue.cc:260] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58701b9c49549838e3d54d37586ef91" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44701 } }
I20260806 16:22:10.492959 11549 raft_consensus.cc:388] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:22:10.493059 11549 raft_consensus.cc:482] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:22:10.493204 11549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:22:10.494886 11549 raft_consensus.cc:504] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58701b9c49549838e3d54d37586ef91" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44701 } }
I20260806 16:22:10.495398 11549 leader_election.cc:302] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d58701b9c49549838e3d54d37586ef91; no voters:
I20260806 16:22:10.495920 11549 leader_election.cc:288] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:22:10.496260 11566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:22:10.496640 11566 raft_consensus.cc:686] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [term 1 LEADER]: Becoming Leader. State: Replica: d58701b9c49549838e3d54d37586ef91, State: Running, Role: LEADER
I20260806 16:22:10.497211 11566 consensus_queue.cc:237] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58701b9c49549838e3d54d37586ef91" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44701 } }
I20260806 16:22:10.498344 11549 sys_catalog.cc:564] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:22:10.500252 11569 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d58701b9c49549838e3d54d37586ef91. Latest consensus state: current_term: 1 leader_uuid: "d58701b9c49549838e3d54d37586ef91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58701b9c49549838e3d54d37586ef91" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44701 } } }
I20260806 16:22:10.500041 11567 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d58701b9c49549838e3d54d37586ef91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58701b9c49549838e3d54d37586ef91" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 44701 } } }
I20260806 16:22:10.500414 11569 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:10.500414 11567 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91 [sys.catalog]: This master's current role is: LEADER
W20260806 16:22:10.507860 11588 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:22:10.507999 11588 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:22:10.508119 11582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:22:10.509131 11582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:22:10.517529 11582 catalog_manager.cc:1347] Generated new cluster ID: 10aa15aeb97543ae8f697e6eb27df576
I20260806 16:22:10.517664 11582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:22:10.530228 11582 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:22:10.531569 11582 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:22:10.537545 11582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d58701b9c49549838e3d54d37586ef91: Generated new TSK 0
I20260806 16:22:10.538717 11582 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:22:10.549437 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 11459
W20260806 16:22:10.572713 11438 connection.cc:535] client connection to 127.11.245.190:44701 recv error: Network error: failed to read from TLS socket (remote: 127.11.245.190:44701): Connection reset by peer (error 104)
2026-08-06T16:22:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1896 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:22:10 dist-test-slave-t3q3 krb5kdc[11599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:22:10 dist-test-slave-t3q3 krb5kdc[11599](info): set up 2 sockets
Aug 06 16:22:10 dist-test-slave-t3q3 krb5kdc[11599](info): commencing operation
krb5kdc: starting...
W20260806 16:22:11.909708 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.001s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:22:12 dist-test-slave-t3q3 krb5kdc[11599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:22:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:22:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:22:12.077329 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:38043
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:35987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:38043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:22:12.245409 11617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:22:12.245899 11617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:22:12.245997 11617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:22:12.251641 11617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:22:12.251804 11617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:22:12.251847 11617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:22:12.251891 11617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:22:12.251927 11617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:22:12.259021 11617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:35987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:38043
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:38043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:22:12.260910 11617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:22:12.262399 11617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:22:12.274662 11624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:12.274973 11629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:12.274922 11617 server_base.cc:1034] running on GCE node
W20260806 16:22:12.274673 11625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:12.275950 11617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:22:12.277820 11617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:22:12.279378 11617 hybrid_clock.cc:648] HybridClock initialized: now 1786033332279340 us; error 52 us; skew 500 ppm
Aug 06 16:22:12 dist-test-slave-t3q3 krb5kdc[11599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033332, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:22:12.283891 11617 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:22:12.285427 11617 webserver.cc:460] Webserver started at http://127.11.245.190:41779/ using document root <none> and password file <none>
I20260806 16:22:12.286434 11617 fs_manager.cc:359] Metadata directory not provided
I20260806 16:22:12.286571 11617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:22:12.286898 11617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:22:12.290057 11617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "d09ef962bd2d46fb89dfc4a464048366"
format_stamp: "Formatted at 2026-08-06 16:22:12 on dist-test-slave-t3q3"
server_key: "feab673f6c7206d6b1cc4a96c43cec58"
server_key_iv: "827790485a0a6e73d27bf1e8398ee918"
server_key_version: "encryptionkey@0"
I20260806 16:22:12.290920 11617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "d09ef962bd2d46fb89dfc4a464048366"
format_stamp: "Formatted at 2026-08-06 16:22:12 on dist-test-slave-t3q3"
server_key: "feab673f6c7206d6b1cc4a96c43cec58"
server_key_iv: "827790485a0a6e73d27bf1e8398ee918"
server_key_version: "encryptionkey@0"
I20260806 16:22:12.296779 11617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:22:12.301244 11637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:22:12.303169 11617 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:22:12.303365 11617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "d09ef962bd2d46fb89dfc4a464048366"
format_stamp: "Formatted at 2026-08-06 16:22:12 on dist-test-slave-t3q3"
server_key: "feab673f6c7206d6b1cc4a96c43cec58"
server_key_iv: "827790485a0a6e73d27bf1e8398ee918"
server_key_version: "encryptionkey@0"
I20260806 16:22:12.303536 11617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:22:12.309768 11617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:22:12.314266 11617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:22:12.314565 11617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:22:12.326615 11617 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:38043
I20260806 16:22:12.326653 11710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:38043 every 8 connection(s)
I20260806 16:22:12.328001 11617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:22:12.332444 11712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:22:12.332435 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 11617
I20260806 16:22:12.332659 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:22:12.333027 12246 external_mini_cluster.cc:1428] Setting key d4814d1546582cfc9be660bcee16c672
I20260806 16:22:12.342020 11715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:12.335037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45113 (local address 127.11.245.190:38043)
0806 16:22:12.335646 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:12.335669 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:12.335725 (+ 56us) server_negotiation.cc:407] Connection header received
0806 16:22:12.336695 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:12.336726 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:12.337214 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:12.337726 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:12.339121 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:12.339146 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:12.339175 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:22:12.339244 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:12.339393 (+ 149us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:22:12.339438 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:12.339834 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:12.339962 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 16:22:12.340241 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":181,"threads_started":1}
I20260806 16:22:12.344754 11712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366: Bootstrap starting.
I20260806 16:22:12.347152 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:22:12.350282 11712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366: Neither blocks nor log segments found. Creating new log.
I20260806 16:22:12.351452 11712 log.cc:824] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366: Log is configured to *not* fsync() on all Append() calls
I20260806 16:22:12.352545 11715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:12.350328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44831 (local address 127.11.245.190:38043)
0806 16:22:12.350528 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:12.350535 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:12.350719 (+ 184us) server_negotiation.cc:407] Connection header received
0806 16:22:12.350928 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:12.350931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:12.350980 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:12.351079 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:12.352012 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:12.352016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:12.352019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:22:12.352032 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:12.352124 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:22:12.352140 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:12.352254 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:12.352329 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 16:22:12.352384 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 16:22:12.354486 11712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366: No bootstrap required, opened a new log
I20260806 16:22:12.359726 11712 raft_consensus.cc:348] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09ef962bd2d46fb89dfc4a464048366" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 38043 } }
I20260806 16:22:12.360198 11712 raft_consensus.cc:374] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:22:12.360253 11712 raft_consensus.cc:729] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d09ef962bd2d46fb89dfc4a464048366, State: Initialized, Role: FOLLOWER
I20260806 16:22:12.360850 11712 consensus_queue.cc:260] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09ef962bd2d46fb89dfc4a464048366" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 38043 } }
I20260806 16:22:12.361011 11712 raft_consensus.cc:388] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:22:12.361107 11712 raft_consensus.cc:482] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:22:12.361224 11712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:22:12.363359 11712 raft_consensus.cc:504] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09ef962bd2d46fb89dfc4a464048366" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 38043 } }
I20260806 16:22:12.364217 11712 leader_election.cc:302] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d09ef962bd2d46fb89dfc4a464048366; no voters:
I20260806 16:22:12.364764 11712 leader_election.cc:288] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:22:12.364957 11727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:22:12.365243 11727 raft_consensus.cc:686] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [term 1 LEADER]: Becoming Leader. State: Replica: d09ef962bd2d46fb89dfc4a464048366, State: Running, Role: LEADER
I20260806 16:22:12.365744 11727 consensus_queue.cc:237] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09ef962bd2d46fb89dfc4a464048366" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 38043 } }
I20260806 16:22:12.366714 11712 sys_catalog.cc:564] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:22:12.368468 11728 sys_catalog.cc:455] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d09ef962bd2d46fb89dfc4a464048366" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09ef962bd2d46fb89dfc4a464048366" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 38043 } } }
I20260806 16:22:12.368644 11728 sys_catalog.cc:458] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:12.368906 11729 sys_catalog.cc:455] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d09ef962bd2d46fb89dfc4a464048366. Latest consensus state: current_term: 1 leader_uuid: "d09ef962bd2d46fb89dfc4a464048366" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09ef962bd2d46fb89dfc4a464048366" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 38043 } } }
I20260806 16:22:12.369153 11729 sys_catalog.cc:458] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366 [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:12.369264 11741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:22:12.373421 11741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:22:12.385207 11741 catalog_manager.cc:1347] Generated new cluster ID: 2cb5322f29ca490f81f9f0b70e0baffc
I20260806 16:22:12.385483 11741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:22:12.404752 11741 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:22:12.405257 11741 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:22:12.414551 11741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d09ef962bd2d46fb89dfc4a464048366: Generated new TSK 0
I20260806 16:22:12.415735 11741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:22:12.431217 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 11617
2026-08-06T16:22:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1882 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:22:12 dist-test-slave-t3q3 krb5kdc[11756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:22:12 dist-test-slave-t3q3 krb5kdc[11756](info): set up 2 sockets
Aug 06 16:22:12 dist-test-slave-t3q3 krb5kdc[11756](info): commencing operation
krb5kdc: starting...
W20260806 16:22:13.783634 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.001s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:22:13 dist-test-slave-t3q3 krb5kdc[11756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:22:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:22:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:22:13.961966 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:34717
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:42927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:34717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:22:14.122551 11779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:22:14.122998 11779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:22:14.123097 11779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:22:14.127799 11779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:22:14.127959 11779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:22:14.127997 11779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:22:14.128039 11779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:22:14.128114 11779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:22:14.135429 11779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:42927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:34717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:34717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:22:14.137256 11779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:22:14.138734 11779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:22:14.150377 11791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:14.150350 11793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:14.150381 11790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:14.150683 11779 server_base.cc:1034] running on GCE node
I20260806 16:22:14.151489 11779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:22:14.153261 11779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:22:14.154647 11779 hybrid_clock.cc:648] HybridClock initialized: now 1786033334154593 us; error 74 us; skew 500 ppm
Aug 06 16:22:14 dist-test-slave-t3q3 krb5kdc[11756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033334, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:22:14.161669 11779 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:22:14.163527 11779 webserver.cc:460] Webserver started at http://127.11.245.190:34953/ using document root <none> and password file <none>
I20260806 16:22:14.164412 11779 fs_manager.cc:359] Metadata directory not provided
I20260806 16:22:14.164494 11779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:22:14.164784 11779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:22:14.167586 11779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "8e39d1e2c2924433acd7cea74cedee21"
format_stamp: "Formatted at 2026-08-06 16:22:14 on dist-test-slave-t3q3"
server_key: "ef2e8ee6e610113dff4789ece1502f42"
server_key_iv: "40612ae339fd785a9080fe9b451b6f04"
server_key_version: "encryptionkey@0"
I20260806 16:22:14.168417 11779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "8e39d1e2c2924433acd7cea74cedee21"
format_stamp: "Formatted at 2026-08-06 16:22:14 on dist-test-slave-t3q3"
server_key: "ef2e8ee6e610113dff4789ece1502f42"
server_key_iv: "40612ae339fd785a9080fe9b451b6f04"
server_key_version: "encryptionkey@0"
I20260806 16:22:14.174553 11779 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 16:22:14.177911 11803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:22:14.179634 11779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:22:14.179841 11779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "8e39d1e2c2924433acd7cea74cedee21"
format_stamp: "Formatted at 2026-08-06 16:22:14 on dist-test-slave-t3q3"
server_key: "ef2e8ee6e610113dff4789ece1502f42"
server_key_iv: "40612ae339fd785a9080fe9b451b6f04"
server_key_version: "encryptionkey@0"
I20260806 16:22:14.180007 11779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:22:14.191856 11779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:22:14.196540 11779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:22:14.196820 11779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:22:14.207055 11779 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:34717
I20260806 16:22:14.207077 11895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:34717 every 8 connection(s)
I20260806 16:22:14.208462 11779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:22:14.212688 11896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:22:14.214563 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 11779
I20260806 16:22:14.214751 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:22:14.215106 12246 external_mini_cluster.cc:1428] Setting key c504a4cccc3a3b17d56da3c6cb7a0568
I20260806 16:22:14.221724 11896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21: Bootstrap starting.
I20260806 16:22:14.224913 11896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21: Neither blocks nor log segments found. Creating new log.
I20260806 16:22:14.225953 11896 log.cc:824] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:22:14 dist-test-slave-t3q3 krb5kdc[11756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:22:14.229094 11896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21: No bootstrap required, opened a new log
I20260806 16:22:14.233856 11901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:14.217162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49445 (local address 127.11.245.190:34717)
0806 16:22:14.217803 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:14.217824 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:14.217883 (+ 59us) server_negotiation.cc:407] Connection header received
0806 16:22:14.218677 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:14.218700 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:14.219363 (+ 663us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:14.219817 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:14.220861 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:14.222020 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:14.222967 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:14.223505 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:14.223688 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:14.227392 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:14.227408 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:14.227427 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:14.227479 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:14.229585 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:14.230260 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:14.230267 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:14.230273 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:14.230421 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:14.230803 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:14.230807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:14.230809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:14.231333 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:14.231511 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:14.232500 (+ 989us) server_negotiation.cc:299] Negotiation successful
0806 16:22:14.232858 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":203,"threads_started":1}
I20260806 16:22:14.234403 11896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e39d1e2c2924433acd7cea74cedee21" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34717 } }
I20260806 16:22:14.234723 11896 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:22:14.234812 11896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e39d1e2c2924433acd7cea74cedee21, State: Initialized, Role: FOLLOWER
I20260806 16:22:14.235456 11896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e39d1e2c2924433acd7cea74cedee21" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34717 } }
I20260806 16:22:14.235651 11896 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:22:14.235711 11896 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:22:14.235813 11896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:22:14.237365 11896 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e39d1e2c2924433acd7cea74cedee21" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34717 } }
I20260806 16:22:14.237876 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:22:14.237848 11896 leader_election.cc:302] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e39d1e2c2924433acd7cea74cedee21; no voters:
I20260806 16:22:14.238279 11896 leader_election.cc:288] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:22:14.238446 11908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:22:14.238698 11908 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [term 1 LEADER]: Becoming Leader. State: Replica: 8e39d1e2c2924433acd7cea74cedee21, State: Running, Role: LEADER
I20260806 16:22:14.239151 11908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e39d1e2c2924433acd7cea74cedee21" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34717 } }
I20260806 16:22:14.240170 11896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:22:14.241552 11911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e39d1e2c2924433acd7cea74cedee21. Latest consensus state: current_term: 1 leader_uuid: "8e39d1e2c2924433acd7cea74cedee21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e39d1e2c2924433acd7cea74cedee21" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34717 } } }
I20260806 16:22:14.241708 11911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:14.242130 11910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e39d1e2c2924433acd7cea74cedee21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e39d1e2c2924433acd7cea74cedee21" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34717 } } }
I20260806 16:22:14.242235 11910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21 [sys.catalog]: This master's current role is: LEADER
W20260806 16:22:14.246445 11937 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:22:14.246630 11937 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:22:14.246743 11926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:22:14.247738 11926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:22:14.254567 11901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:14.241493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60309 (local address 127.11.245.190:34717)
0806 16:22:14.241636 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:14.241642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:14.242979 (+ 1337us) server_negotiation.cc:407] Connection header received
0806 16:22:14.243232 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:14.243236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:14.243294 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:14.243396 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:14.244607 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:14.245462 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:14.246849 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:14.247371 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:14.247536 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:14.249573 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:14.249578 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:14.249581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:14.249616 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:14.251835 (+ 2219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:14.252468 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:14.252473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:14.252475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:14.252586 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:14.252941 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:14.252945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:14.252947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:14.253256 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:14.253412 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:14.253826 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 16:22:14.254403 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 16:22:14.258503 11926 catalog_manager.cc:1347] Generated new cluster ID: 27ee4ed7805e4541bf8735018ffe6bcc
I20260806 16:22:14.258656 11926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:22:14.278739 11926 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:22:14.280303 11926 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:22:14.292479 11926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e39d1e2c2924433acd7cea74cedee21: Generated new TSK 0
I20260806 16:22:14.293601 11926 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:22:14.304378 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 11779
2026-08-06T16:22:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1867 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:22:14 dist-test-slave-t3q3 krb5kdc[11949](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:22:14 dist-test-slave-t3q3 krb5kdc[11949](info): set up 2 sockets
Aug 06 16:22:14 dist-test-slave-t3q3 krb5kdc[11949](info): commencing operation
krb5kdc: starting...
W20260806 16:22:15.653064 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.000s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:22:15 dist-test-slave-t3q3 krb5kdc[11949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:22:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:22:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:22:15.829097 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:34725
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:34725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:22:15.986835 11966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:22:15.987236 11966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:22:15.987291 11966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:22:15.992682 11966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:22:15.992785 11966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:22:15.992812 11966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:22:15.992832 11966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:22:15.992851 11966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:22:16.000168 11966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:34725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:34725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:22:16.002034 11966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:22:16.003604 11966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:22:16.015993 11974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:16.016155 11966 server_base.cc:1034] running on GCE node
W20260806 16:22:16.015868 11977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:16.015858 11975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:16.017205 11966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:22:16.018817 11966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:22:16.020097 11966 hybrid_clock.cc:648] HybridClock initialized: now 1786033336020045 us; error 41 us; skew 500 ppm
Aug 06 16:22:16 dist-test-slave-t3q3 krb5kdc[11949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033336, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:22:16.023948 11966 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:22:16.025626 11966 webserver.cc:460] Webserver started at http://127.11.245.190:40099/ using document root <none> and password file <none>
I20260806 16:22:16.026445 11966 fs_manager.cc:359] Metadata directory not provided
I20260806 16:22:16.026563 11966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:22:16.026878 11966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:22:16.029742 11966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "de613d40319749a4b92e4716f6705896"
format_stamp: "Formatted at 2026-08-06 16:22:16 on dist-test-slave-t3q3"
server_key: "24d3ed3972da32a5c2a4729e28d1578a"
server_key_iv: "c1d603d0934f5646e8b36557a640b026"
server_key_version: "encryptionkey@0"
I20260806 16:22:16.030622 11966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "de613d40319749a4b92e4716f6705896"
format_stamp: "Formatted at 2026-08-06 16:22:16 on dist-test-slave-t3q3"
server_key: "24d3ed3972da32a5c2a4729e28d1578a"
server_key_iv: "c1d603d0934f5646e8b36557a640b026"
server_key_version: "encryptionkey@0"
I20260806 16:22:16.035934 11966 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:22:16.039592 11986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:22:16.042068 11966 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:22:16.042379 11966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "de613d40319749a4b92e4716f6705896"
format_stamp: "Formatted at 2026-08-06 16:22:16 on dist-test-slave-t3q3"
server_key: "24d3ed3972da32a5c2a4729e28d1578a"
server_key_iv: "c1d603d0934f5646e8b36557a640b026"
server_key_version: "encryptionkey@0"
I20260806 16:22:16.042553 11966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:22:16.062263 11966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:22:16.067265 11966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:22:16.067600 11966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:22:16.077198 11966 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:34725
I20260806 16:22:16.077219 12065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:34725 every 8 connection(s)
I20260806 16:22:16.078574 11966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:22:16.081768 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 11966
I20260806 16:22:16.081964 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:22:16.082333 12246 external_mini_cluster.cc:1428] Setting key 0ef9c71358f0188fe88e58b402fb7da0
I20260806 16:22:16.082816 12066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:22:16.092772 12066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896: Bootstrap starting.
I20260806 16:22:16.095455 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:16.084310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49621 (local address 127.11.245.190:34725)
0806 16:22:16.084909 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:16.084934 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:16.084998 (+ 64us) server_negotiation.cc:407] Connection header received
0806 16:22:16.085975 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:16.086006 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:16.086578 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:16.087067 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:16.087630 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:16.089397 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:16.090490 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:16.091326 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:16.091513 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:16.092782 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:16.092791 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:16.092815 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:22:16.093469 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:16.093553 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:22:16.093590 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:16.093709 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:16.093923 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 16:22:16.094223 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":168,"threads_started":1}
I20260806 16:22:16.096464 12066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896: Neither blocks nor log segments found. Creating new log.
I20260806 16:22:16.097431 12066 log.cc:824] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896: Log is configured to *not* fsync() on all Append() calls
I20260806 16:22:16.099332 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:22:16.100240 12066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896: No bootstrap required, opened a new log
I20260806 16:22:16.105644 12066 raft_consensus.cc:348] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de613d40319749a4b92e4716f6705896" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34725 } }
I20260806 16:22:16.106112 12066 raft_consensus.cc:374] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:22:16.106207 12066 raft_consensus.cc:729] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de613d40319749a4b92e4716f6705896, State: Initialized, Role: FOLLOWER
I20260806 16:22:16.106952 12066 consensus_queue.cc:260] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de613d40319749a4b92e4716f6705896" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34725 } }
I20260806 16:22:16.107153 12066 raft_consensus.cc:388] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:22:16.107271 12066 raft_consensus.cc:482] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:22:16.107422 12066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:22:16.107872 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:16.102275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37921 (local address 127.11.245.190:34725)
0806 16:22:16.102431 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:16.102439 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:16.102483 (+ 44us) server_negotiation.cc:407] Connection header received
0806 16:22:16.102643 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:16.102647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:16.102704 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:16.102798 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:16.103120 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:16.104611 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:16.105588 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:16.105939 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:16.106070 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:16.107266 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:16.107271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:16.107274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:22:16.107297 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:16.107335 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:22:16.107351 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:16.107437 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:16.107642 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 16:22:16.107721 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 16:22:16.109752 12066 raft_consensus.cc:504] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de613d40319749a4b92e4716f6705896" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34725 } }
I20260806 16:22:16.110332 12066 leader_election.cc:302] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de613d40319749a4b92e4716f6705896; no voters:
I20260806 16:22:16.110744 12066 leader_election.cc:288] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:22:16.111133 12085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:22:16.111413 12085 raft_consensus.cc:686] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [term 1 LEADER]: Becoming Leader. State: Replica: de613d40319749a4b92e4716f6705896, State: Running, Role: LEADER
I20260806 16:22:16.111984 12085 consensus_queue.cc:237] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de613d40319749a4b92e4716f6705896" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34725 } }
I20260806 16:22:16.112811 12066 sys_catalog.cc:564] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:22:16.114845 12086 sys_catalog.cc:455] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de613d40319749a4b92e4716f6705896" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de613d40319749a4b92e4716f6705896" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34725 } } }
I20260806 16:22:16.115043 12086 sys_catalog.cc:458] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:16.115289 12087 sys_catalog.cc:455] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de613d40319749a4b92e4716f6705896. Latest consensus state: current_term: 1 leader_uuid: "de613d40319749a4b92e4716f6705896" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de613d40319749a4b92e4716f6705896" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 34725 } } }
I20260806 16:22:16.115401 12087 sys_catalog.cc:458] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896 [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:16.115568 12101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:22:16.120441 12101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:22:16.129696 12101 catalog_manager.cc:1347] Generated new cluster ID: 07a51dba49a24b7ca3be260232398371
I20260806 16:22:16.129839 12101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:22:16.149241 12101 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:22:16.151196 12101 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:22:16.173650 12101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de613d40319749a4b92e4716f6705896: Generated new TSK 0
I20260806 16:22:16.174822 12101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:22:16.183024 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 11966
2026-08-06T16:22:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1877 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:22:16 dist-test-slave-t3q3 krb5kdc[12113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:22:16 dist-test-slave-t3q3 krb5kdc[12113](info): set up 2 sockets
Aug 06 16:22:16 dist-test-slave-t3q3 krb5kdc[12113](info): commencing operation
krb5kdc: starting...
W20260806 16:22:17.568218 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.351s user 0.001s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:22:17 dist-test-slave-t3q3 krb5kdc[12113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:22:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:22:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:22:17.749249 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:45907
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:39465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:45907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:22:17.909868 12128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:22:17.910271 12128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:22:17.910329 12128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:22:17.914861 12128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:22:17.914963 12128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:22:17.914983 12128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:22:17.915002 12128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:22:17.915019 12128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:22:17.921893 12128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:39465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:45907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:45907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:22:17.923736 12128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:22:17.925307 12128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:22:17.938369 12138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:17.938355 12135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:17.938355 12134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:17.938813 12128 server_base.cc:1034] running on GCE node
I20260806 16:22:17.939657 12128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:22:17.941433 12128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:22:17.942683 12128 hybrid_clock.cc:648] HybridClock initialized: now 1786033337942651 us; error 53 us; skew 500 ppm
Aug 06 16:22:17 dist-test-slave-t3q3 krb5kdc[12113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033337, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:22:17.946604 12128 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:22:17.948262 12128 webserver.cc:460] Webserver started at http://127.11.245.190:45293/ using document root <none> and password file <none>
I20260806 16:22:17.949064 12128 fs_manager.cc:359] Metadata directory not provided
I20260806 16:22:17.949174 12128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:22:17.949484 12128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:22:17.952387 12128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "f884bf471f3c415f88e350264bfc24aa"
format_stamp: "Formatted at 2026-08-06 16:22:17 on dist-test-slave-t3q3"
server_key: "dbb673fce15a7d8dea14daf30e152db7"
server_key_iv: "b6b2a9336c1f73de504dfe5e27c5cdc2"
server_key_version: "encryptionkey@0"
I20260806 16:22:17.953191 12128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "f884bf471f3c415f88e350264bfc24aa"
format_stamp: "Formatted at 2026-08-06 16:22:17 on dist-test-slave-t3q3"
server_key: "dbb673fce15a7d8dea14daf30e152db7"
server_key_iv: "b6b2a9336c1f73de504dfe5e27c5cdc2"
server_key_version: "encryptionkey@0"
I20260806 16:22:17.958263 12128 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:22:17.961740 12147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:22:17.963771 12128 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:22:17.963974 12128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "f884bf471f3c415f88e350264bfc24aa"
format_stamp: "Formatted at 2026-08-06 16:22:17 on dist-test-slave-t3q3"
server_key: "dbb673fce15a7d8dea14daf30e152db7"
server_key_iv: "b6b2a9336c1f73de504dfe5e27c5cdc2"
server_key_version: "encryptionkey@0"
I20260806 16:22:17.964193 12128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:22:17.970232 12128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:22:17.974714 12128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:22:17.974999 12128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:22:17.984411 12128 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:45907
I20260806 16:22:17.984444 12223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:45907 every 8 connection(s)
I20260806 16:22:17.985733 12128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:22:17.990082 12224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:22:17.992187 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 12128
I20260806 16:22:17.992396 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:22:17.992807 12246 external_mini_cluster.cc:1428] Setting key f19c59d6cb7057a7c03ef0d9243f079d
I20260806 16:22:17.999915 12224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa: Bootstrap starting.
I20260806 16:22:18.000890 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:17.995104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42711 (local address 127.11.245.190:45907)
0806 16:22:17.995815 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:17.995835 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:17.995906 (+ 71us) server_negotiation.cc:407] Connection header received
0806 16:22:17.996835 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:17.996859 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:17.997268 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:17.997718 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:17.998795 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:17.998800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:17.998815 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:22:17.998827 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:17.998911 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:22:17.998934 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:17.999140 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:17.999227 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 16:22:17.999541 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":233,"threads_started":1}
I20260806 16:22:18.003475 12224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa: Neither blocks nor log segments found. Creating new log.
I20260806 16:22:18.004756 12224 log.cc:824] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa: Log is configured to *not* fsync() on all Append() calls
I20260806 16:22:18.006143 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:22:18.007754 12224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa: No bootstrap required, opened a new log
I20260806 16:22:18.011726 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:18.009282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35593 (local address 127.11.245.190:45907)
0806 16:22:18.009441 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:18.009445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:18.009800 (+ 355us) server_negotiation.cc:407] Connection header received
0806 16:22:18.009902 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:18.009905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:18.009974 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:18.010118 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:18.011080 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:18.011083 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:18.011086 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:22:18.011100 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:18.011136 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:22:18.011148 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:18.011242 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:18.011441 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 16:22:18.011525 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:22:18.014313 12224 raft_consensus.cc:348] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f884bf471f3c415f88e350264bfc24aa" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45907 } }
I20260806 16:22:18.014688 12224 raft_consensus.cc:374] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:22:18.014766 12224 raft_consensus.cc:729] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f884bf471f3c415f88e350264bfc24aa, State: Initialized, Role: FOLLOWER
I20260806 16:22:18.015408 12224 consensus_queue.cc:260] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f884bf471f3c415f88e350264bfc24aa" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45907 } }
I20260806 16:22:18.015620 12224 raft_consensus.cc:388] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:22:18.015728 12224 raft_consensus.cc:482] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:22:18.016016 12224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:22:18.017844 12224 raft_consensus.cc:504] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f884bf471f3c415f88e350264bfc24aa" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45907 } }
I20260806 16:22:18.018370 12224 leader_election.cc:302] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f884bf471f3c415f88e350264bfc24aa; no voters:
I20260806 16:22:18.018968 12224 leader_election.cc:288] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:22:18.019160 12241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:22:18.019441 12241 raft_consensus.cc:686] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [term 1 LEADER]: Becoming Leader. State: Replica: f884bf471f3c415f88e350264bfc24aa, State: Running, Role: LEADER
I20260806 16:22:18.019933 12241 consensus_queue.cc:237] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f884bf471f3c415f88e350264bfc24aa" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45907 } }
I20260806 16:22:18.020911 12224 sys_catalog.cc:564] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:22:18.022348 12244 sys_catalog.cc:455] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader f884bf471f3c415f88e350264bfc24aa. Latest consensus state: current_term: 1 leader_uuid: "f884bf471f3c415f88e350264bfc24aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f884bf471f3c415f88e350264bfc24aa" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45907 } } }
I20260806 16:22:18.022344 12242 sys_catalog.cc:455] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f884bf471f3c415f88e350264bfc24aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f884bf471f3c415f88e350264bfc24aa" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45907 } } }
I20260806 16:22:18.022543 12244 sys_catalog.cc:458] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:18.022547 12242 sys_catalog.cc:458] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:18.023238 12263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:22:18.027586 12263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:22:18.036449 12263 catalog_manager.cc:1347] Generated new cluster ID: eac35cbe145449b397ff0b6805ad2191
I20260806 16:22:18.036567 12263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:22:18.073858 12263 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:22:18.074749 12263 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:22:18.087131 12263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f884bf471f3c415f88e350264bfc24aa: Generated new TSK 0
I20260806 16:22:18.088637 12263 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:22:18.103497 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 12128
2026-08-06T16:22:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1928 ms)
[----------] 18 tests from AuthTokenIssuingTest (33976 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:22:18 dist-test-slave-t3q3 krb5kdc[12282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:22:18 dist-test-slave-t3q3 krb5kdc[12282](info): set up 2 sockets
Aug 06 16:22:18 dist-test-slave-t3q3 krb5kdc[12282](info): commencing operation
krb5kdc: starting...
W20260806 16:22:19.516707 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.371s user 0.003s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:22:19 dist-test-slave-t3q3 krb5kdc[12282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:22:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:22:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:22:19.701004 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:45159
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:44677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:45159,127.11.245.189:33245,127.11.245.188:36595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:22:19.871716 12303 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:22:19.872316 12303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:22:19.872382 12303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:22:19.872433 12303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:22:19.877221 12303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:22:19.877321 12303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:22:19.877344 12303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:22:19.877363 12303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:22:19.877382 12303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:22:19.884483 12303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:44677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:45159,127.11.245.189:33245,127.11.245.188:36595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:45159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:22:19.887176 12303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:22:19.888788 12303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:22:19.901029 12312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:19.901005 12314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:19.901136 12311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:19.902235 12303 server_base.cc:1034] running on GCE node
I20260806 16:22:19.903153 12303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:22:19.904943 12303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:22:19.906178 12303 hybrid_clock.cc:648] HybridClock initialized: now 1786033339906148 us; error 54 us; skew 500 ppm
Aug 06 16:22:19 dist-test-slave-t3q3 krb5kdc[12282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033339, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:22:19.909910 12303 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:22:19.911206 12303 webserver.cc:460] Webserver started at http://127.11.245.190:43939/ using document root <none> and password file <none>
I20260806 16:22:19.912042 12303 fs_manager.cc:359] Metadata directory not provided
I20260806 16:22:19.912215 12303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:22:19.912542 12303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:22:19.916019 12303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "9197a4091b044e28805698986390404b"
format_stamp: "Formatted at 2026-08-06 16:22:19 on dist-test-slave-t3q3"
server_key: "e8266473a2f5d3e25a6c83994c6efdb4"
server_key_iv: "b585e0a6cd6d38edab46c978fc5bf010"
server_key_version: "encryptionkey@0"
I20260806 16:22:19.916982 12303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "9197a4091b044e28805698986390404b"
format_stamp: "Formatted at 2026-08-06 16:22:19 on dist-test-slave-t3q3"
server_key: "e8266473a2f5d3e25a6c83994c6efdb4"
server_key_iv: "b585e0a6cd6d38edab46c978fc5bf010"
server_key_version: "encryptionkey@0"
I20260806 16:22:19.923559 12303 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 16:22:19.927354 12329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:22:19.929486 12303 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:22:19.929808 12303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "9197a4091b044e28805698986390404b"
format_stamp: "Formatted at 2026-08-06 16:22:19 on dist-test-slave-t3q3"
server_key: "e8266473a2f5d3e25a6c83994c6efdb4"
server_key_iv: "b585e0a6cd6d38edab46c978fc5bf010"
server_key_version: "encryptionkey@0"
I20260806 16:22:19.930006 12303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:22:19.954137 12303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:22:19.959074 12303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:22:19.959384 12303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:22:19.970413 12303 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:45159
I20260806 16:22:19.970765 12406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:45159 every 8 connection(s)
I20260806 16:22:19.972004 12303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:22:19.974829 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 12303
I20260806 16:22:19.975064 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:22:19.975551 12246 external_mini_cluster.cc:1428] Setting key c20c4e5988dff9c87046a9b36644d79e
I20260806 16:22:19.978396 12407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:22:19.986128 12407 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.245.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.189@KRBTEST.COM" created.
Aug 06 16:22:19 dist-test-slave-t3q3 krb5kdc[12282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033339, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:22:20.011924 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:19.989858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:45159 (local address 127.0.0.1:39548)
0806 16:22:19.990765 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:19.990837 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:19.992592 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:19.993733 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:19.993740 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:19.994344 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:19.994658 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:19.994675 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:19.996007 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:19.996010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:19.996901 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:19.996920 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:19.997693 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:19.997697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:19.997796 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:19.998561 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:19.998582 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:20.002003 (+ 3421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:20.006835 (+ 4832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.006855 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.006877 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.007379 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.008271 (+ 892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.008277 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.008280 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.008498 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.009656 (+ 1158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:20.009664 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:20.009855 (+ 191us) client_negotiation.cc:769] Sending connection context
0806 16:22:20.010068 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 16:22:20.010652 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":113,"thread_start_us":353,"threads_started":1}
I20260806 16:22:20.011930 12412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:19.989858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39548 (local address 127.11.245.190:45159)
0806 16:22:19.990569 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:19.990596 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:19.990988 (+ 392us) server_negotiation.cc:407] Connection header received
0806 16:22:19.993152 (+ 2164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:19.993159 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:19.993461 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:19.993734 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:19.994842 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:19.995841 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:19.997043 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:19.997490 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:19.997680 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.002626 (+ 4946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:20.002638 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:20.002658 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:20.002699 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:20.006547 (+ 3848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.007814 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.007826 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.007828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.008113 (+ 285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.008658 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.008661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.008663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.009475 (+ 812us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:20.009623 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:20.010244 (+ 621us) server_negotiation.cc:299] Negotiation successful
0806 16:22:20.010568 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":494,"spinlock_wait_cycles":126080,"thread_start_us":277,"threads_started":1}
I20260806 16:22:20.015516 12407 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.189.keytab.
Entry for principal kudu/127.11.245.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.189.keytab.
I20260806 16:22:20.016949 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.016477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:33245 (local address 127.0.0.1:48020)
0806 16:22:20.016670 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.016834 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260806 16:22:20.017212 12337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.245.189:33245: connect: Connection refused (error 111)
I20260806 16:22:20.018159 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.189:33245
--webserver_interface=127.11.245.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.189.keytab
--principal=kudu/127.11.245.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:44677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:45159,127.11.245.189:33245,127.11.245.188:36595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:22:20.018782 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.018257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:33245 (local address 127.0.0.1:48028)
0806 16:22:20.018596 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.018673 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209}
W20260806 16:22:20.019008 12407 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.189:33245: Network error: Client connection negotiation failed: client connection to 127.11.245.189:33245: connect: Connection refused (error 111)
I20260806 16:22:20.068410 12407 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } attempt: 1
I20260806 16:22:20.069986 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.069543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:33245 (local address 127.0.0.1:48032)
0806 16:22:20.069763 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.069874 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260806 16:22:20.070950 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.070699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:33245 (local address 127.0.0.1:48044)
0806 16:22:20.070829 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.070889 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 16:22:20.071149 12407 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.189:33245: Network error: Client connection negotiation failed: client connection to 127.11.245.189:33245: connect: Connection refused (error 111)
I20260806 16:22:20.139381 12407 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } attempt: 2
I20260806 16:22:20.140566 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.140173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:33245 (local address 127.0.0.1:48050)
0806 16:22:20.140384 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.140465 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 16:22:20.141839 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.141518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:33245 (local address 127.0.0.1:48052)
0806 16:22:20.141693 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.141767 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260806 16:22:20.142119 12407 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.189:33245: Network error: Client connection negotiation failed: client connection to 127.11.245.189:33245: connect: Connection refused (error 111)
W20260806 16:22:20.182199 12419 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:22:20.182650 12419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:22:20.182680 12419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:22:20.182797 12419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:22:20.187417 12419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:22:20.187536 12419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:22:20.187557 12419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:22:20.187575 12419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:22:20.187592 12419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:22:20.194551 12419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:44677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:45159,127.11.245.189:33245,127.11.245.188:36595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.189.keytab
--principal=kudu/127.11.245.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.189:33245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.245.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:22:20.196388 12419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:22:20.197865 12419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:22:20.210413 12435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:20.210548 12438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:20.210413 12433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:20.210793 12419 server_base.cc:1034] running on GCE node
I20260806 16:22:20.211969 12419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:22:20.213879 12419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:22:20.215433 12419 hybrid_clock.cc:648] HybridClock initialized: now 1786033340215410 us; error 60 us; skew 500 ppm
Aug 06 16:22:20 dist-test-slave-t3q3 krb5kdc[12282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033340, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:22:20.219635 12419 init.cc:377] Logged in from keytab as kudu/127.11.245.189@KRBTEST.COM (short username kudu)
I20260806 16:22:20.221263 12419 webserver.cc:460] Webserver started at http://127.11.245.189:40797/ using document root <none> and password file <none>
I20260806 16:22:20.222075 12419 fs_manager.cc:359] Metadata directory not provided
I20260806 16:22:20.222186 12419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:22:20.222502 12419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:22:20.225389 12419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/data/instance:
uuid: "5c5e210cd9d3420687b81f81b7bb53e2"
format_stamp: "Formatted at 2026-08-06 16:22:20 on dist-test-slave-t3q3"
server_key: "5205e9fda302dc23898cbcfa40eef7af"
server_key_iv: "9cf894d14ee10ad63904d92580748bb1"
server_key_version: "encryptionkey@0"
I20260806 16:22:20.226120 12419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/wal/instance:
uuid: "5c5e210cd9d3420687b81f81b7bb53e2"
format_stamp: "Formatted at 2026-08-06 16:22:20 on dist-test-slave-t3q3"
server_key: "5205e9fda302dc23898cbcfa40eef7af"
server_key_iv: "9cf894d14ee10ad63904d92580748bb1"
server_key_version: "encryptionkey@0"
I20260806 16:22:20.231448 12419 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:22:20.233345 12407 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } attempt: 3
I20260806 16:22:20.234608 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.234135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:33245 (local address 127.0.0.1:48058)
0806 16:22:20.234392 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.234474 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 16:22:20.235437 12451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:22:20.235771 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.235379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:33245 (local address 127.0.0.1:48060)
0806 16:22:20.235555 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.235679 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260806 16:22:20.236016 12407 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.189:33245: Network error: Client connection negotiation failed: client connection to 127.11.245.189:33245: connect: Connection refused (error 111)
I20260806 16:22:20.237314 12419 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:22:20.237517 12419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/wal
uuid: "5c5e210cd9d3420687b81f81b7bb53e2"
format_stamp: "Formatted at 2026-08-06 16:22:20 on dist-test-slave-t3q3"
server_key: "5205e9fda302dc23898cbcfa40eef7af"
server_key_iv: "9cf894d14ee10ad63904d92580748bb1"
server_key_version: "encryptionkey@0"
I20260806 16:22:20.237679 12419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:22:20.262827 12419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:22:20.267226 12419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:22:20.267630 12419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:22:20.277410 12419 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.189:33245
I20260806 16:22:20.277899 12531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.189:33245 every 8 connection(s)
I20260806 16:22:20.278877 12419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/data/info.pb
I20260806 16:22:20.281883 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 12419
I20260806 16:22:20.282063 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-1/wal/instance
I20260806 16:22:20.282430 12246 external_mini_cluster.cc:1428] Setting key 782fc3d78928f609a3a696d06ac4dd85
I20260806 16:22:20.283440 12537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:22:20.291061 12537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.245.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.188@KRBTEST.COM" created.
Aug 06 16:22:20 dist-test-slave-t3q3 krb5kdc[12282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033340, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.189@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:22:20.312687 12412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.294106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39552 (local address 127.11.245.190:45159)
0806 16:22:20.294290 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.294295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.295798 (+ 1503us) server_negotiation.cc:407] Connection header received
0806 16:22:20.297391 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.297396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.297477 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.297575 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.299269 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.300054 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.302251 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.302554 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.302665 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.307121 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:20.307126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:20.307128 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:20.307168 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:20.309215 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.310229 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.310233 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.310235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.310340 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.310911 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.310914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.310915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.311199 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:20.311288 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:20.312319 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0806 16:22:20.312512 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 16:22:20.314200 12545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.294787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:45159 (local address 127.0.0.1:39552)
0806 16:22:20.295592 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.295646 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:20.297116 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:20.297836 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:20.297845 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:20.298343 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:20.299093 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:20.299110 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.300298 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.300302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:20.302059 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:20.302078 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.302771 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.302778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:20.303065 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.304491 (+ 1426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:20.304522 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:20.306782 (+ 2260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:20.309410 (+ 2628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.309427 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.309464 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.310011 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.310466 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.310472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.310476 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.310748 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.311330 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:20.311343 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:20.311797 (+ 454us) client_negotiation.cc:769] Sending connection context
0806 16:22:20.312039 (+ 242us) client_negotiation.cc:240] Negotiation successful
0806 16:22:20.312563 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":250,"threads_started":1}
I20260806 16:22:20.316828 12537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.188.keytab.
Entry for principal kudu/127.11.245.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.188.keytab.
Aug 06 16:22:20 dist-test-slave-t3q3 krb5kdc[12282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033340, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.189@KRBTEST.COM for kudu/127.11.245.189@KRBTEST.COM
I20260806 16:22:20.325078 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.188:36595
--webserver_interface=127.11.245.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.188.keytab
--principal=kudu/127.11.245.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:44677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:45159,127.11.245.189:33245,127.11.245.188:36595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:22:20.329137 12545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.317744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:33245 (local address 127.0.0.1:48070)
0806 16:22:20.317919 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.317935 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:20.318072 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:20.318893 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:20.318897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:20.319177 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:20.319312 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:20.319319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.320636 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.320639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:20.321348 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:20.321358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.322135 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.322138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:20.322247 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.322926 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:20.322952 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:20.324605 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:20.327093 (+ 2488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.327098 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.327101 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.327316 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.327643 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.327648 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.327650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.327734 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.328422 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:20.328425 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:20.328534 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 16:22:20.328661 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 16:22:20.328887 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 16:22:20.329149 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.317838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48070 (local address 127.11.245.189:33245)
0806 16:22:20.318281 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.318294 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.318318 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:22:20.318358 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.318363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.318607 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.318941 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.319500 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.320495 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.321471 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.321981 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.322112 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.324778 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:20.324790 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:20.324810 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:20.324831 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:20.326941 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.327450 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.327463 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.327464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.327544 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.327828 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.327833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.327835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.328301 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:20.328485 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:20.328783 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 16:22:20.328943 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":153,"threads_started":1}
I20260806 16:22:20.332805 12537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:22:20.333992 12545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.333603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47314)
0806 16:22:20.333775 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.333884 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 16:22:20.334254 12455 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
I20260806 16:22:20.335413 12545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.335112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47328)
0806 16:22:20.335279 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.335333 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 16:22:20.335615 12537 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:36595: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
I20260806 16:22:20.379319 12407 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } attempt: 4
I20260806 16:22:20.384999 12537 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } attempt: 1
I20260806 16:22:20.386051 12545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.385661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47342)
0806 16:22:20.385834 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.385913 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 16:22:20.387352 12545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.387053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47354)
0806 16:22:20.387195 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.387257 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 16:22:20.387615 12537 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:36595: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
Aug 06 16:22:20 dist-test-slave-t3q3 krb5kdc[12282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033339, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for kudu/127.11.245.189@KRBTEST.COM
I20260806 16:22:20.392024 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.380331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48072 (local address 127.11.245.189:33245)
0806 16:22:20.380567 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.380572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.380666 (+ 94us) server_negotiation.cc:407] Connection header received
0806 16:22:20.380881 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.380885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.380952 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.381039 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.382226 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.382835 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.384287 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.385342 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.385491 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.388473 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:20.388477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:20.388479 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:20.388508 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:20.390129 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.390632 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.390637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.390639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.390728 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.391038 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.391042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.391044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.391280 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:20.391405 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:20.391707 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 16:22:20.391845 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260806 16:22:20.392025 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.380275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:33245 (local address 127.0.0.1:48072)
0806 16:22:20.380558 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.380579 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:20.380726 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:20.381093 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:20.381096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:20.381359 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:20.382058 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:20.382070 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.382984 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.382988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:20.383865 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:20.383877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.385531 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.385539 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:20.385707 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.386471 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:20.386489 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:20.388292 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:20.390273 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.390277 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.390280 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.390520 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.390831 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.390834 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.390836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.390938 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.391385 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:20.391389 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:20.391523 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 16:22:20.391639 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 16:22:20.391837 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260806 16:22:20.393376 12407 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:22:20.394279 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.393998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47366)
0806 16:22:20.394134 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.394196 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 16:22:20.394984 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.394759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47376)
0806 16:22:20.394865 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.394912 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 16:22:20.395108 12407 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:36595: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
I20260806 16:22:20.454367 12407 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } attempt: 1
I20260806 16:22:20.455751 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.455373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47390)
0806 16:22:20.455566 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.455648 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 16:22:20.455822 12537 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } attempt: 2
I20260806 16:22:20.456669 12545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.456370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47392)
0806 16:22:20.456512 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.456579 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 16:22:20.456734 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.456438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47402)
0806 16:22:20.456612 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.456665 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260806 16:22:20.457123 12407 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:36595: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
I20260806 16:22:20.457394 12545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.457187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47414)
0806 16:22:20.457279 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.457339 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 16:22:20.457553 12537 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:36595: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
W20260806 16:22:20.486660 12550 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:22:20.487169 12550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:22:20.487241 12550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:22:20.487351 12550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:22:20.493130 12550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:22:20.493314 12550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:22:20.493356 12550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:22:20.493407 12550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:22:20.493471 12550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:22:20.501740 12550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:44677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:45159,127.11.245.189:33245,127.11.245.188:36595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.188.keytab
--principal=kudu/127.11.245.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.188:36595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.245.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:22:20.503517 12550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:22:20.505018 12550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:22:20.517009 12562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:20.517009 12559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:20.517000 12560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:20.517639 12550 server_base.cc:1034] running on GCE node
I20260806 16:22:20.518675 12550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:22:20.520855 12550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:22:20.522099 12550 hybrid_clock.cc:648] HybridClock initialized: now 1786033340522079 us; error 39 us; skew 500 ppm
I20260806 16:22:20.524302 12407 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } attempt: 2
I20260806 16:22:20.525460 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.525017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47418)
0806 16:22:20.525241 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.525328 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
Aug 06 16:22:20 dist-test-slave-t3q3 krb5kdc[12282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033340, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:22:20.526268 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.526037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47426)
0806 16:22:20.526150 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.526200 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 16:22:20.526510 12550 init.cc:377] Logged in from keytab as kudu/127.11.245.188@KRBTEST.COM (short username kudu)
W20260806 16:22:20.526585 12407 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:36595: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
I20260806 16:22:20.527868 12550 webserver.cc:460] Webserver started at http://127.11.245.188:33987/ using document root <none> and password file <none>
I20260806 16:22:20.528715 12550 fs_manager.cc:359] Metadata directory not provided
I20260806 16:22:20.528792 12550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:22:20.529102 12550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:22:20.532009 12550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/data/instance:
uuid: "8c73359bf085457788f3f4f1dafa0b35"
format_stamp: "Formatted at 2026-08-06 16:22:20 on dist-test-slave-t3q3"
server_key: "9c38b5a3a015804e5bd714e476af8e6b"
server_key_iv: "088c362eefc0a8b2a9368d5b60558b87"
server_key_version: "encryptionkey@0"
I20260806 16:22:20.532954 12550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/wal/instance:
uuid: "8c73359bf085457788f3f4f1dafa0b35"
format_stamp: "Formatted at 2026-08-06 16:22:20 on dist-test-slave-t3q3"
server_key: "9c38b5a3a015804e5bd714e476af8e6b"
server_key_iv: "088c362eefc0a8b2a9368d5b60558b87"
server_key_version: "encryptionkey@0"
I20260806 16:22:20.538425 12550 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 16:22:20.541954 12570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:22:20.544271 12550 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:22:20.544476 12550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/wal
uuid: "8c73359bf085457788f3f4f1dafa0b35"
format_stamp: "Formatted at 2026-08-06 16:22:20 on dist-test-slave-t3q3"
server_key: "9c38b5a3a015804e5bd714e476af8e6b"
server_key_iv: "088c362eefc0a8b2a9368d5b60558b87"
server_key_version: "encryptionkey@0"
I20260806 16:22:20.544658 12550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:22:20.548738 12537 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } attempt: 3
I20260806 16:22:20.549904 12545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.549515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47434)
0806 16:22:20.549712 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.549786 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 16:22:20.551066 12545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.550687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47444)
0806 16:22:20.550891 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.550975 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260806 16:22:20.551263 12537 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:36595: Network error: Client connection negotiation failed: client connection to 127.11.245.188:36595: connect: Connection refused (error 111)
I20260806 16:22:20.564489 12550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:22:20.569060 12550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:22:20.569294 12550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:22:20.581312 12550 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.188:36595
I20260806 16:22:20.581686 12654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.188:36595 every 8 connection(s)
I20260806 16:22:20.582866 12550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/data/info.pb
I20260806 16:22:20.587049 12655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:22:20.588635 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 12550
I20260806 16:22:20.588789 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786033011181791-12246-0/minicluster-data/master-2/wal/instance
I20260806 16:22:20.589175 12246 external_mini_cluster.cc:1428] Setting key b6129f898a3faa6471fd3ece5c85a441
I20260806 16:22:20.593304 12655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:22:20 dist-test-slave-t3q3 krb5kdc[12282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.188@KRBTEST.COM
Aug 06 16:22:20 dist-test-slave-t3q3 krb5kdc[12282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033340, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.188@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:22:20.610534 12412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.595606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39566 (local address 127.11.245.190:45159)
0806 16:22:20.595797 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.595803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.596518 (+ 715us) server_negotiation.cc:407] Connection header received
0806 16:22:20.596700 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.596703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.596757 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.596852 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.597725 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.598393 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.599855 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.600219 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.600356 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.603525 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:20.603530 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:20.603533 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:20.603576 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:20.606837 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.608331 (+ 1494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.608335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.608336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.608488 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.609044 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.609050 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.609054 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.609410 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:20.609528 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:20.610049 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 16:22:20.610289 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 16:22:20.611756 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.595522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:45159 (local address 127.0.0.1:39566)
0806 16:22:20.596332 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.596393 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:20.596561 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:20.596951 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:20.596957 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:20.597428 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:20.597573 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:20.597580 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.598540 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.598544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:20.599691 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:20.599704 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.600424 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.600430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:20.600571 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.601195 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:20.601214 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:20.603246 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:20.607603 (+ 4357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.607700 (+ 97us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.607722 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.608160 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.608652 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.608656 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.608658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.608909 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.609542 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:20.609549 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:20.609722 (+ 173us) client_negotiation.cc:769] Sending connection context
0806 16:22:20.609973 (+ 251us) client_negotiation.cc:240] Negotiation successful
0806 16:22:20.610498 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":217,"threads_started":1}
I20260806 16:22:20.612380 12662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.591162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47460 (local address 127.11.245.188:36595)
0806 16:22:20.591851 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.591873 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.591926 (+ 53us) server_negotiation.cc:407] Connection header received
0806 16:22:20.592883 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.592908 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.593393 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.593853 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.595087 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.596972 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.598128 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.598861 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.599050 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.602860 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:20.602872 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:20.602892 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:20.602926 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:20.607365 (+ 4439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.608253 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.608299 (+ 46us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.608302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.608502 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.608955 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.608958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.608960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.609714 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:20.609963 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:20.610247 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 16:22:20.612232 (+ 1985us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1732,"server-negotiator.queue_time_us":485,"thread_start_us":245,"threads_started":1}
I20260806 16:22:20.613584 12655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:22:20 dist-test-slave-t3q3 krb5kdc[12282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033340, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.188@KRBTEST.COM for kudu/127.11.245.189@KRBTEST.COM
Aug 06 16:22:20 dist-test-slave-t3q3 krb5kdc[12282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:22:20.626911 12407 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } attempt: 3
I20260806 16:22:20.627460 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.614228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:33245 (local address 127.0.0.1:48078)
0806 16:22:20.614374 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.614389 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:20.614484 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:20.614890 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:20.614893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:20.615079 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:20.615291 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:20.615304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.616528 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.616531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:20.617199 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:20.617211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.618018 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.618025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:20.618171 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.618978 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:20.619000 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:20.621473 (+ 2473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:20.624710 (+ 3237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.624713 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.624715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.624956 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.625358 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.625360 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.625362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.625452 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.626487 (+ 1035us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:20.626493 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:20.626625 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 16:22:20.626726 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 16:22:20.627225 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 16:22:20.627535 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.614419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48078 (local address 127.11.245.189:33245)
0806 16:22:20.614585 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.614590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.614606 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:22:20.614663 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.614667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.614727 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.614825 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.615467 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.616384 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.617372 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.617780 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.617895 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.621723 (+ 3828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:20.621727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:20.621730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:20.621973 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:20.624532 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.625077 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.625084 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.625088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.625203 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.625748 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.625755 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.625757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.626341 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:20.626461 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:20.626937 (+ 476us) server_negotiation.cc:299] Negotiation successful
0806 16:22:20.627207 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 16:22:20.628787 12412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.615593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39570 (local address 127.11.245.190:45159)
0806 16:22:20.615730 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.615734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.615751 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:22:20.616111 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.616118 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.616185 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.616265 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.617123 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.617990 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.619223 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.619570 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.619717 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.623340 (+ 3623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:20.623349 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:20.623357 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:20.623408 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:20.625418 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.626068 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.626072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.626074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.626190 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.626585 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.626587 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.626588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.627099 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:20.628039 (+ 940us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:20.628435 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 16:22:20.628605 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 16:22:20.628863 12655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:22:20 dist-test-slave-t3q3 krb5kdc[12282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033339, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for kudu/127.11.245.188@KRBTEST.COM
Aug 06 16:22:20 dist-test-slave-t3q3 krb5kdc[12282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033340, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.188@KRBTEST.COM for kudu/127.11.245.188@KRBTEST.COM
Aug 06 16:22:20 dist-test-slave-t3q3 krb5kdc[12282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.189@KRBTEST.COM
I20260806 16:22:20.639906 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.627709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47476)
0806 16:22:20.628427 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.628445 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:20.628534 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:20.628847 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:20.628851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:20.629279 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:20.629464 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:20.629473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.630669 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.630673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:20.631399 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:20.631434 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.632236 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.632240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:20.632337 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.632984 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:20.633000 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:20.634874 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:20.637463 (+ 2589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.637469 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.637472 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.637806 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.638172 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.638176 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.638178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.638290 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.639069 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:20.639074 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:20.639327 (+ 253us) client_negotiation.cc:769] Sending connection context
0806 16:22:20.639482 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 16:22:20.639683 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626}
I20260806 16:22:20.639911 12662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.628117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47476 (local address 127.11.245.188:36595)
0806 16:22:20.628260 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.628265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.628631 (+ 366us) server_negotiation.cc:407] Connection header received
0806 16:22:20.628674 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.628677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.628726 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.629611 (+ 885us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.629740 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.630542 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.631566 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.631971 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.632097 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.635062 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:20.635067 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:20.635070 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:20.635101 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:20.637296 (+ 2195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.637924 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.637929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.637931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.638041 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.638397 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.638403 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.638405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.638715 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:20.639094 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:20.639574 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 16:22:20.639769 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 16:22:20.642010 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.629889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47482)
0806 16:22:20.630710 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.630727 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:20.630808 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:20.631490 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:20.631497 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:20.631717 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:20.631810 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:20.631816 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.632942 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.632948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:20.633772 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:20.633783 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.634278 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.634281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:20.634413 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.635006 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:20.635019 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:20.637073 (+ 2054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:20.640202 (+ 3129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.640206 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.640210 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.640465 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.640774 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.640778 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.640781 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.640853 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.641342 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:20.641345 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:20.641464 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 16:22:20.641575 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 16:22:20.641799 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":42}
I20260806 16:22:20.642616 12664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.630653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47482 (local address 127.11.245.188:36595)
0806 16:22:20.631235 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.631248 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.631262 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:22:20.631297 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.631300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.631341 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.631448 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.631944 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.632795 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.633906 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.634158 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.634271 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.637256 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:20.637262 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:20.637265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:20.637298 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:20.639991 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.640579 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.640585 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.640588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.640678 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.640928 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.640932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.640933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.641223 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:20.641332 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:20.642229 (+ 897us) server_negotiation.cc:299] Negotiation successful
0806 16:22:20.642443 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":510,"thread_start_us":186,"threads_started":1}
I20260806 16:22:20.643814 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.631028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48086 (local address 127.11.245.189:33245)
0806 16:22:20.631193 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.631198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.631215 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:22:20.631281 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.631285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.631342 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.632579 (+ 1237us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.632740 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.633446 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.634673 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.635112 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.635240 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.638683 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:20.638687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:20.638690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:20.638719 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:20.640835 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.641472 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.641475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.641477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.641575 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.641907 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.641911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.641913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.642313 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:20.643393 (+ 1080us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:20.643504 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 16:22:20.643651 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 16:22:20.645304 12407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b: Bootstrap starting.
I20260806 16:22:20.646682 12655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35: Bootstrap starting.
I20260806 16:22:20.648307 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:22:20.648744 12407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b: Neither blocks nor log segments found. Creating new log.
I20260806 16:22:20.649776 12407 log.cc:824] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b: Log is configured to *not* fsync() on all Append() calls
I20260806 16:22:20.649904 12655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35: Neither blocks nor log segments found. Creating new log.
I20260806 16:22:20.650985 12655 log.cc:824] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35: Log is configured to *not* fsync() on all Append() calls
I20260806 16:22:20.650141 12371 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9197a4091b044e28805698986390404b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39570
I20260806 16:22:20.652591 12407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b: No bootstrap required, opened a new log
I20260806 16:22:20.653463 12655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35: No bootstrap required, opened a new log
I20260806 16:22:20.655586 12407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } }
I20260806 16:22:20.655861 12407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9197a4091b044e28805698986390404b, State: Initialized, Role: FOLLOWER
I20260806 16:22:20.656330 12407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } }
I20260806 16:22:20.656944 12668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } } }
I20260806 16:22:20.657099 12668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:22:20.658000 12407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:22:20.658085 12371 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 16:22:20.658236 12371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:22:20.658257 12655 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } }
I20260806 16:22:20.658547 12655 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c73359bf085457788f3f4f1dafa0b35, State: Initialized, Role: FOLLOWER
I20260806 16:22:20.659052 12655 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } }
I20260806 16:22:20.659698 12678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } } }
I20260806 16:22:20.659965 12678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:22:20.660287 12371 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } }
I20260806 16:22:20.660921 12655 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:22:20.664235 12371 leader_election.cc:288] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [CANDIDATE]: Term 1 election: Requested vote from peers 5c5e210cd9d3420687b81f81b7bb53e2 (127.11.245.189:33245), 8c73359bf085457788f3f4f1dafa0b35 (127.11.245.188:36595)
W20260806 16:22:20.665932 12691 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:22:20.666165 12691 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:22:20.663790 12616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9197a4091b044e28805698986390404b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8c73359bf085457788f3f4f1dafa0b35"
I20260806 16:22:20.668339 12616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:22:20.670789 12616 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9197a4091b044e28805698986390404b in term 1.
I20260806 16:22:20.671396 12337 leader_election.cc:302] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c73359bf085457788f3f4f1dafa0b35, 9197a4091b044e28805698986390404b; no voters:
I20260806 16:22:20.671877 12668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:22:20.672186 12668 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [term 1 LEADER]: Becoming Leader. State: Replica: 9197a4091b044e28805698986390404b, State: Running, Role: LEADER
I20260806 16:22:20.672701 12668 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } }
I20260806 16:22:20.666325 12303 master_runner.cc:186] Error getting master registration for 127.11.245.189:33245: OK, instance_id { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" instance_seqno: 1786033340274716 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:22:20.673941 12303 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:22:20.670064 12550 master_runner.cc:186] Error getting master registration for 127.11.245.189:33245: OK, instance_id { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" instance_seqno: 1786033340274716 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:22:20.674278 12550 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:22:20.675384 12668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9197a4091b044e28805698986390404b. Latest consensus state: current_term: 1 leader_uuid: "9197a4091b044e28805698986390404b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } } }
I20260806 16:22:20.675572 12668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [sys.catalog]: This master's current role is: LEADER
W20260806 16:22:20.675608 12709 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:22:20.675830 12709 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:22:20.676255 12723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:22:20.677505 12723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:22:20.687867 12616 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35 [term 1 FOLLOWER]: Refusing update from remote peer 9197a4091b044e28805698986390404b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:22:20.692240 12718 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:22:20.694427 12537 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } attempt: 4
I20260806 16:22:20.699051 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.668039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48094 (local address 127.11.245.189:33245)
0806 16:22:20.668236 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.668243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.676346 (+ 8103us) server_negotiation.cc:407] Connection header received
0806 16:22:20.684687 (+ 8341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.684692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.684775 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.684867 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.686629 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.687413 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.689135 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.689496 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.689588 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.693141 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:20.693146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:20.693149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:20.693195 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:20.695886 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.696781 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.696786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.696788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.696893 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.697458 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.697461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.697463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.697745 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:20.697885 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:20.698559 (+ 674us) server_negotiation.cc:299] Negotiation successful
0806 16:22:20.698713 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 16:22:20.699283 12664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.668988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47484 (local address 127.11.245.188:36595)
0806 16:22:20.669139 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.669144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.676659 (+ 7515us) server_negotiation.cc:407] Connection header received
0806 16:22:20.684705 (+ 8046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.684709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.684778 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.684843 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.686428 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.687331 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.688784 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.689189 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.689316 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.693468 (+ 4152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:20.693474 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:20.693477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:20.693878 (+ 401us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:20.696485 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.697302 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.697307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.697309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.697421 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.697927 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.697931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.697933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.698185 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:20.698311 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:20.698958 (+ 647us) server_negotiation.cc:299] Negotiation successful
0806 16:22:20.699111 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:22:20.701263 12678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9197a4091b044e28805698986390404b. Latest consensus state: current_term: 1 leader_uuid: "9197a4091b044e28805698986390404b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } } }
I20260806 16:22:20.701473 12678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35 [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 16:22:20 dist-test-slave-t3q3 krb5kdc[12282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033340, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.189@KRBTEST.COM for kudu/127.11.245.188@KRBTEST.COM
I20260806 16:22:20.709825 12545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.694976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:36595 (local address 127.0.0.1:47492)
0806 16:22:20.695234 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:20.695252 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:20.695374 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:20.695860 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:20.695863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:20.696095 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:20.696258 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:20.696268 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.697747 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.697752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:20.698654 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:20.698667 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.699483 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.699487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:20.699596 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.700220 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:20.700236 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:20.702244 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:20.704696 (+ 2452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.704702 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.704704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.705245 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.705917 (+ 672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:20.705921 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:20.705923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:20.706183 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:20.707231 (+ 1048us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:20.707235 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:20.707492 (+ 257us) client_negotiation.cc:769] Sending connection context
0806 16:22:20.707759 (+ 267us) client_negotiation.cc:240] Negotiation successful
0806 16:22:20.708610 (+ 851us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260806 16:22:20.709758 12662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.695084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47492 (local address 127.11.245.188:36595)
0806 16:22:20.695234 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.695240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.695306 (+ 66us) server_negotiation.cc:407] Connection header received
0806 16:22:20.695478 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.695482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.695569 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.695667 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.696420 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.697302 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.698877 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.699209 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.699329 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.702401 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:20.702406 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:20.702409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:20.702436 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:20.704519 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.705437 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.705444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.705480 (+ 36us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.705708 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.706387 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.706391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.706393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.707014 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:20.707841 (+ 827us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:20.708257 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 16:22:20.709062 (+ 805us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 16:22:20.711956 12339 tablet.cc:2369] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:22:20.713868 12412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:20.669491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39586 (local address 127.11.245.190:45159)
0806 16:22:20.669623 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.669631 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.689063 (+ 19432us) server_negotiation.cc:407] Connection header received
0806 16:22:20.689277 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.689280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.689343 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.689408 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.702570 (+ 13162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.703514 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.704852 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:20.705186 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:20.705305 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:20.707351 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:20.707356 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:20.707360 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:20.707396 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:20.709722 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.710459 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.710463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.710466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.710573 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:20.710913 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:20.710917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:20.710919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:20.711208 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:20.711393 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:20.712908 (+ 1515us) server_negotiation.cc:299] Negotiation successful
0806 16:22:20.713470 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 16:22:20.715726 12537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2: Bootstrap starting.
I20260806 16:22:20.722110 12537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2: Neither blocks nor log segments found. Creating new log.
I20260806 16:22:20.723459 12537 log.cc:824] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2: Log is configured to *not* fsync() on all Append() calls
I20260806 16:22:20.729993 12537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2: No bootstrap required, opened a new log
I20260806 16:22:20.735265 12718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9197a4091b044e28805698986390404b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } } }
I20260806 16:22:20.735692 12718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:20.740208 12678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9197a4091b044e28805698986390404b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } } }
I20260806 16:22:20.740432 12678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:22:20.743659 12723 catalog_manager.cc:1347] Generated new cluster ID: 35e654cdfe50404abb8b0e7d6ec56534
I20260806 16:22:20.743772 12723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:22:20.749517 12537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } }
I20260806 16:22:20.750177 12537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5e210cd9d3420687b81f81b7bb53e2, State: Initialized, Role: FOLLOWER
I20260806 16:22:20.750751 12537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } }
I20260806 16:22:20.753050 12537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:22:20.753737 12490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:22:20.755911 12490 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2 [term 1 FOLLOWER]: Refusing update from remote peer 9197a4091b044e28805698986390404b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:22:20.756157 12494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9197a4091b044e28805698986390404b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5c5e210cd9d3420687b81f81b7bb53e2"
I20260806 16:22:20.759156 12494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9197a4091b044e28805698986390404b in term 1.
I20260806 16:22:20.761332 12738 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:22:20.769827 12761 mvcc.cc:205] Tried to move back new op lower bound from 7315592563442384896 to 7315592563400679424. Current Snapshot: MvccSnapshot[applied={T|T < 7315592563442384896}]
I20260806 16:22:20.776328 12763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9197a4091b044e28805698986390404b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } } }
I20260806 16:22:20.776657 12763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:22:20.776787 12763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9197a4091b044e28805698986390404b. Latest consensus state: current_term: 1 leader_uuid: "9197a4091b044e28805698986390404b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } } }
I20260806 16:22:20.776897 12763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:22:20.777004 12763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9197a4091b044e28805698986390404b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } } }
I20260806 16:22:20.777107 12763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:22:20.780920 12718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9197a4091b044e28805698986390404b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9197a4091b044e28805698986390404b" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 45159 } } peers { permanent_uuid: "5c5e210cd9d3420687b81f81b7bb53e2" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 33245 } } peers { permanent_uuid: "8c73359bf085457788f3f4f1dafa0b35" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 36595 } } }
I20260806 16:22:20.781113 12718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:20.783486 12723 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:22:20.785279 12723 catalog_manager.cc:1504] Loading token signing keys...
W20260806 16:22:20.793846 12758 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:22:20.795698 12758 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:22:20.804199 12723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9197a4091b044e28805698986390404b: Generated new TSK 0
I20260806 16:22:20.805485 12723 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 16:22:20.856122 12549 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:20.851943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48096 (local address 127.11.245.189:33245)
0806 16:22:20.852207 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.852213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.854581 (+ 2368us) server_negotiation.cc:407] Connection header received
0806 16:22:20.854836 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.854841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.854914 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.854984 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.855864 (+ 880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48096: BlockingRecv error: recv got EOF from 127.0.0.1:48096 (error 108)
Metrics: {"server-negotiator.queue_time_us":150}
W20260806 16:22:20.858076 12662 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:20.853065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47504 (local address 127.11.245.188:36595)
0806 16:22:20.853207 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.853214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.856615 (+ 3401us) server_negotiation.cc:407] Connection header received
0806 16:22:20.856863 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.856867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.856938 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.856997 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.857892 (+ 895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47504: BlockingRecv error: recv got EOF from 127.0.0.1:47504 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 16:22:20.859402 12412 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:20.853931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39602 (local address 127.11.245.190:45159)
0806 16:22:20.854069 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.854074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.858496 (+ 4422us) server_negotiation.cc:407] Connection header received
0806 16:22:20.858618 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.858622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.858692 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.858779 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.859251 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39602: BlockingRecv error: recv got EOF from 127.0.0.1:39602 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 16:22:20.865528 12549 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:20.863150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48098 (local address 127.11.245.189:33245)
0806 16:22:20.863310 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.863315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.864162 (+ 847us) server_negotiation.cc:407] Connection header received
0806 16:22:20.864468 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.864472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.864590 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.864667 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.865366 (+ 699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48098: BlockingRecv error: recv got EOF from 127.0.0.1:48098 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 16:22:20.873347 12549 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:20.870528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48102 (local address 127.11.245.189:33245)
0806 16:22:20.870680 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.870685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.871690 (+ 1005us) server_negotiation.cc:407] Connection header received
0806 16:22:20.871977 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.871982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.872055 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.872222 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.873163 (+ 941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48102: BlockingRecv error: recv got EOF from 127.0.0.1:48102 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 16:22:20.881222 12549 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:20.878617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48104 (local address 127.11.245.189:33245)
0806 16:22:20.878975 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.878981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.879786 (+ 805us) server_negotiation.cc:407] Connection header received
0806 16:22:20.880175 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.880180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.880253 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.880321 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.881052 (+ 731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48104: BlockingRecv error: recv got EOF from 127.0.0.1:48104 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 16:22:20.890919 12549 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:20.888367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48120 (local address 127.11.245.189:33245)
0806 16:22:20.888537 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.888542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.889376 (+ 834us) server_negotiation.cc:407] Connection header received
0806 16:22:20.889800 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.889805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.889888 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.889956 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.890727 (+ 771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48120: BlockingRecv error: recv got EOF from 127.0.0.1:48120 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 16:22:20.904875 12549 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:20.902444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48128 (local address 127.11.245.189:33245)
0806 16:22:20.902627 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.902633 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.903492 (+ 859us) server_negotiation.cc:407] Connection header received
0806 16:22:20.903772 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.903777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.903852 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.903921 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.904710 (+ 789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48128: BlockingRecv error: recv got EOF from 127.0.0.1:48128 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 16:22:20.926899 12549 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:20.924438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48138 (local address 127.11.245.189:33245)
0806 16:22:20.924624 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.924630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.925505 (+ 875us) server_negotiation.cc:407] Connection header received
0806 16:22:20.925781 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.925786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.925872 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.925939 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.926694 (+ 755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48138: BlockingRecv error: recv got EOF from 127.0.0.1:48138 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260806 16:22:20.964658 12549 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:20.962472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48144 (local address 127.11.245.189:33245)
0806 16:22:20.962668 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:20.962674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:20.963111 (+ 437us) server_negotiation.cc:407] Connection header received
0806 16:22:20.963438 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:20.963442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:20.963519 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:20.963585 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:20.964491 (+ 906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48144: BlockingRecv error: recv got EOF from 127.0.0.1:48144 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 16:22:21.033200 12549 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:21.031557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48148 (local address 127.11.245.189:33245)
0806 16:22:21.031720 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:21.031724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:21.031992 (+ 268us) server_negotiation.cc:407] Connection header received
0806 16:22:21.032123 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:21.032126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:21.032190 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:21.032285 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:21.032991 (+ 706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48148: BlockingRecv error: recv got EOF from 127.0.0.1:48148 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 16:22:21.167035 12549 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:21.165377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48156 (local address 127.11.245.189:33245)
0806 16:22:21.165568 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:21.165572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:21.165587 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:22:21.165656 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:21.165659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:21.165715 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:21.165816 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:21.166735 (+ 919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48156: BlockingRecv error: recv got EOF from 127.0.0.1:48156 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 16:22:21.427964 12549 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:21.426748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48168 (local address 127.11.245.189:33245)
0806 16:22:21.426928 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:21.426933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:21.426951 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:22:21.427022 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:21.427024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:21.427083 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:21.427162 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:21.427766 (+ 604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48168: BlockingRecv error: recv got EOF from 127.0.0.1:48168 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 16:22:21.677971 12709 catalog_manager.cc:1259] Loaded cluster ID: 35e654cdfe50404abb8b0e7d6ec56534
I20260806 16:22:21.678145 12709 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35: loading cluster ID for follower catalog manager: success
I20260806 16:22:21.680323 12709 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35: acquiring CA information for follower catalog manager: success
I20260806 16:22:21.681867 12709 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8c73359bf085457788f3f4f1dafa0b35: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:22:21.797653 12758 catalog_manager.cc:1259] Loaded cluster ID: 35e654cdfe50404abb8b0e7d6ec56534
I20260806 16:22:21.797788 12758 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2: loading cluster ID for follower catalog manager: success
I20260806 16:22:21.799925 12758 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2: acquiring CA information for follower catalog manager: success
I20260806 16:22:21.801523 12758 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5c5e210cd9d3420687b81f81b7bb53e2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:22:21.948539 12867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:21.943270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48178 (local address 127.11.245.189:33245)
0806 16:22:21.943601 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:21.943605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:21.943624 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:22:21.943792 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:21.943795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:21.944022 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:21.944202 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:22:21.944582 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:21.945248 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:21.946257 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:21.946542 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:21.946645 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:21.947067 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:22:21.947751 (+ 684us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:22:21.947874 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:21.948144 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 16:22:21.948280 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":136,"threads_started":1}
I20260806 16:22:21.957604 12876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:21.952944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47514 (local address 127.11.245.188:36595)
0806 16:22:21.953296 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:21.953307 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:21.953326 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:22:21.953521 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:21.953524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:21.953689 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:21.953888 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:22:21.954142 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:21.954728 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:21.955668 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:21.955965 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:21.956103 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:21.956482 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:22:21.957124 (+ 642us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:22:21.957277 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:21.957324 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 16:22:21.957401 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":94,"threads_started":1}
I20260806 16:22:21.967571 12867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:21.962378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48188 (local address 127.11.245.189:33245)
0806 16:22:21.962560 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:21.962564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:21.963208 (+ 644us) server_negotiation.cc:407] Connection header received
0806 16:22:21.963526 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:21.963529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:21.963637 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:21.963730 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:22:21.964020 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:21.964933 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:21.966186 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:21.966514 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:21.966624 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:21.966945 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:22:21.967059 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:22:21.967110 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:21.967323 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 16:22:21.967400 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 16:22:21.967581 12876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:21.963017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47516 (local address 127.11.245.188:36595)
0806 16:22:21.963162 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:21.963166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:21.963283 (+ 117us) server_negotiation.cc:407] Connection header received
0806 16:22:21.963761 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:21.963765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:21.963824 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:21.963875 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:22:21.964308 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:21.965085 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:21.966350 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:21.966714 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:21.966826 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:21.967109 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:22:21.967185 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:22:21.967269 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:21.967359 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 16:22:21.967428 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 16:22:21.969177 12885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:21.962307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39614 (local address 127.11.245.190:45159)
0806 16:22:21.963749 (+ 1442us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:21.963753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:21.963768 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:22:21.963900 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:21.963904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:21.964132 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:21.964300 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:22:21.964650 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:21.965328 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:21.966426 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:21.966743 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:21.966853 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:21.967356 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:22:21.968781 (+ 1425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:22:21.968885 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:21.968960 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 16:22:21.969036 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1321,"thread_start_us":80,"threads_started":1}
I20260806 16:22:21.973251 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 12303
I20260806 16:22:21.996527 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 12419
I20260806 16:22:22.017186 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 12550
2026-08-06T16:22:22Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3911 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:22:22 dist-test-slave-t3q3 krb5kdc[12891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:22:22 dist-test-slave-t3q3 krb5kdc[12891](info): set up 2 sockets
Aug 06 16:22:22 dist-test-slave-t3q3 krb5kdc[12891](info): commencing operation
krb5kdc: starting...
W20260806 16:22:23.361737 12246 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/test-user.keytab.
Aug 06 16:22:23 dist-test-slave-t3q3 krb5kdc[12891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:22:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:22:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.245.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
Entry for principal kudu/127.11.245.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab.
I20260806 16:22:23.545796 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:41333
--webserver_interface=127.11.245.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:44517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:41333,127.11.245.189:44055,127.11.245.188:34953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:22:23.709937 12913 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:22:23.710359 12913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:22:23.710386 12913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:22:23.710494 12913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:22:23.715695 12913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:22:23.715838 12913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:22:23.715862 12913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:22:23.715884 12913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:22:23.715903 12913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:22:23.723850 12913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:44517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:41333,127.11.245.189:44055,127.11.245.188:34953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.190.keytab
--principal=kudu/127.11.245.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.190:41333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.245.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:22:23.725692 12913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:22:23.727753 12913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:22:23.739930 12922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:23.740007 12923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:23.739972 12927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:23.740456 12913 server_base.cc:1034] running on GCE node
I20260806 16:22:23.741178 12913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:22:23.742671 12913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:22:23.744004 12913 hybrid_clock.cc:648] HybridClock initialized: now 1786033343743954 us; error 57 us; skew 500 ppm
Aug 06 16:22:23 dist-test-slave-t3q3 krb5kdc[12891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:22:23.748454 12913 init.cc:377] Logged in from keytab as kudu/127.11.245.190@KRBTEST.COM (short username kudu)
I20260806 16:22:23.749807 12913 webserver.cc:460] Webserver started at http://127.11.245.190:40601/ using document root <none> and password file <none>
I20260806 16:22:23.750662 12913 fs_manager.cc:359] Metadata directory not provided
I20260806 16:22:23.750774 12913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:22:23.751076 12913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:22:23.753993 12913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/data/instance:
uuid: "836458b37909437886c0e17ac02f42e8"
format_stamp: "Formatted at 2026-08-06 16:22:23 on dist-test-slave-t3q3"
server_key: "d60f1fc4e64a537767c391977942daa3"
server_key_iv: "b354aa2c0b3a56c7e4f709d60fd521e2"
server_key_version: "encryptionkey@0"
I20260806 16:22:23.754753 12913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/wal/instance:
uuid: "836458b37909437886c0e17ac02f42e8"
format_stamp: "Formatted at 2026-08-06 16:22:23 on dist-test-slave-t3q3"
server_key: "d60f1fc4e64a537767c391977942daa3"
server_key_iv: "b354aa2c0b3a56c7e4f709d60fd521e2"
server_key_version: "encryptionkey@0"
I20260806 16:22:23.760346 12913 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 16:22:23.764953 12939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:22:23.767252 12913 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:22:23.767534 12913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/wal
uuid: "836458b37909437886c0e17ac02f42e8"
format_stamp: "Formatted at 2026-08-06 16:22:23 on dist-test-slave-t3q3"
server_key: "d60f1fc4e64a537767c391977942daa3"
server_key_iv: "b354aa2c0b3a56c7e4f709d60fd521e2"
server_key_version: "encryptionkey@0"
I20260806 16:22:23.767733 12913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:22:23.793339 12913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:22:23.799367 12913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:22:23.799675 12913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:22:23.811776 12913 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.190:41333
I20260806 16:22:23.812534 13019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.190:41333 every 8 connection(s)
I20260806 16:22:23.813809 12913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/data/info.pb
I20260806 16:22:23.818157 13020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:22:23.820539 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 12913
I20260806 16:22:23.821003 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-0/wal/instance
I20260806 16:22:23.821962 12246 external_mini_cluster.cc:1428] Setting key fc2535eecc60795d4de9bbbd5368f089
I20260806 16:22:23.826828 13020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.245.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.189@KRBTEST.COM" created.
Aug 06 16:22:23 dist-test-slave-t3q3 krb5kdc[12891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:22:23.854045 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:23.832226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:41333 (local address 127.0.0.1:46770)
0806 16:22:23.833193 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:23.833258 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:23.834562 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:23.835985 (+ 1423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:23.835991 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:23.836702 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:23.837220 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:23.837247 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:23.839018 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:23.839026 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:23.840288 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:23.840301 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:23.841096 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:23.841100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:23.841263 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:23.842228 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:23.842259 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:23.845877 (+ 3618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:23.849447 (+ 3570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:23.849459 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:23.849476 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:23.849792 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:23.850399 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:23.850402 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:23.850404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:23.850556 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:23.851937 (+ 1381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:23.851945 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:23.852203 (+ 258us) client_negotiation.cc:769] Sending connection context
0806 16:22:23.852407 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 16:22:23.852821 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":6,"spinlock_wait_cycles":89856,"thread_start_us":248,"threads_started":1}
I20260806 16:22:23.854050 13025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:23.832226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46770 (local address 127.11.245.190:41333)
0806 16:22:23.833032 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:23.833058 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:23.833437 (+ 379us) server_negotiation.cc:407] Connection header received
0806 16:22:23.835177 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:23.835183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:23.835676 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:23.835923 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:23.837519 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:23.838863 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:23.840445 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:23.840900 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:23.841101 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:23.846283 (+ 5182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:23.846292 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:23.846315 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:23.846375 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:23.849164 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:23.849996 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:23.850013 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:23.850018 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:23.850225 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:23.850803 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:23.850811 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:23.850816 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:23.851730 (+ 914us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:23.851941 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:23.852569 (+ 628us) server_negotiation.cc:299] Negotiation successful
0806 16:22:23.852946 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":182,"server-negotiator.queue_time_us":546,"thread_start_us":228,"threads_started":1}
I20260806 16:22:23.858439 13020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:22:23.859793 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:23.859337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:44055 (local address 127.0.0.1:45318)
0806 16:22:23.859572 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:23.859678 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260806 16:22:23.860198 12944 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.245.189:44055: connect: Connection refused (error 111)
I20260806 16:22:23.861789 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:23.861508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:44055 (local address 127.0.0.1:45332)
0806 16:22:23.861645 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:23.861711 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 16:22:23.861986 13020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.189:44055: Network error: Client connection negotiation failed: client connection to 127.11.245.189:44055: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.189.keytab.
Entry for principal kudu/127.11.245.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.189.keytab.
I20260806 16:22:23.873914 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.189:44055
--webserver_interface=127.11.245.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.189.keytab
--principal=kudu/127.11.245.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:44517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:41333,127.11.245.189:44055,127.11.245.188:34953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:22:23.911583 13020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } attempt: 1
I20260806 16:22:23.912989 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:23.912507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:44055 (local address 127.0.0.1:45338)
0806 16:22:23.912755 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:23.912872 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260806 16:22:23.914016 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:23.913747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:44055 (local address 127.0.0.1:45350)
0806 16:22:23.913878 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:23.913947 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 16:22:23.914155 13020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.189:44055: Network error: Client connection negotiation failed: client connection to 127.11.245.189:44055: connect: Connection refused (error 111)
I20260806 16:22:23.982389 13020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } attempt: 2
I20260806 16:22:23.983752 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:23.983266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:44055 (local address 127.0.0.1:45354)
0806 16:22:23.983551 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:23.983640 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260806 16:22:23.984817 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:23.984536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:44055 (local address 127.0.0.1:45360)
0806 16:22:23.984677 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:23.984738 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 16:22:23.984997 13020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.189:44055: Network error: Client connection negotiation failed: client connection to 127.11.245.189:44055: connect: Connection refused (error 111)
W20260806 16:22:24.044981 13031 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:22:24.045655 13031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:22:24.045750 13031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:22:24.045970 13031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:22:24.052925 13031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:22:24.053087 13031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:22:24.053128 13031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:22:24.053171 13031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:22:24.053213 13031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:22:24.060434 13031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:44517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:41333,127.11.245.189:44055,127.11.245.188:34953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.189.keytab
--principal=kudu/127.11.245.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.189:44055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.245.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:22:24.062172 13031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:22:24.063832 13031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:22:24.076167 13020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } attempt: 3
W20260806 16:22:24.076435 13045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:24.076395 13041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:24.076725 13031 server_base.cc:1034] running on GCE node
W20260806 16:22:24.076997 13038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:24.077302 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.076911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:44055 (local address 127.0.0.1:45362)
0806 16:22:24.077120 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.077199 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 16:22:24.078274 13031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:22:24.078382 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.078071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:44055 (local address 127.0.0.1:45374)
0806 16:22:24.078226 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.078294 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.189:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 16:22:24.078608 13020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.189:44055: Network error: Client connection negotiation failed: client connection to 127.11.245.189:44055: connect: Connection refused (error 111)
I20260806 16:22:24.080020 13031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:22:24.081274 13031 hybrid_clock.cc:648] HybridClock initialized: now 1786033344081262 us; error 47 us; skew 500 ppm
Aug 06 16:22:24 dist-test-slave-t3q3 krb5kdc[12891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033344, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:22:24.085878 13031 init.cc:377] Logged in from keytab as kudu/127.11.245.189@KRBTEST.COM (short username kudu)
I20260806 16:22:24.087268 13031 webserver.cc:460] Webserver started at http://127.11.245.189:37633/ using document root <none> and password file <none>
I20260806 16:22:24.088310 13031 fs_manager.cc:359] Metadata directory not provided
I20260806 16:22:24.088464 13031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:22:24.088757 13031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:22:24.091753 13031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/data/instance:
uuid: "1757dee06d4f446daf2e11085b417a6c"
format_stamp: "Formatted at 2026-08-06 16:22:24 on dist-test-slave-t3q3"
server_key: "2ae0e8115dedd6fa1e31f7a1746f8a67"
server_key_iv: "bd4965ccaec8ac9e5003e061efedf9d3"
server_key_version: "encryptionkey@0"
I20260806 16:22:24.092609 13031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/wal/instance:
uuid: "1757dee06d4f446daf2e11085b417a6c"
format_stamp: "Formatted at 2026-08-06 16:22:24 on dist-test-slave-t3q3"
server_key: "2ae0e8115dedd6fa1e31f7a1746f8a67"
server_key_iv: "bd4965ccaec8ac9e5003e061efedf9d3"
server_key_version: "encryptionkey@0"
I20260806 16:22:24.098919 13031 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 16:22:24.105114 13056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:22:24.107631 13031 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.004s
I20260806 16:22:24.107858 13031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/wal
uuid: "1757dee06d4f446daf2e11085b417a6c"
format_stamp: "Formatted at 2026-08-06 16:22:24 on dist-test-slave-t3q3"
server_key: "2ae0e8115dedd6fa1e31f7a1746f8a67"
server_key_iv: "bd4965ccaec8ac9e5003e061efedf9d3"
server_key_version: "encryptionkey@0"
I20260806 16:22:24.108031 13031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:22:24.123289 13031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:22:24.129719 13031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:22:24.129984 13031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:22:24.140995 13031 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.189:44055
I20260806 16:22:24.141434 13139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.189:44055 every 8 connection(s)
I20260806 16:22:24.143325 13031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/data/info.pb
I20260806 16:22:24.147737 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 13031
I20260806 16:22:24.147876 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-1/wal/instance
I20260806 16:22:24.148342 12246 external_mini_cluster.cc:1428] Setting key 00cac23b77c7fcd0341bdd8b5e45a04d
I20260806 16:22:24.148766 13141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:22:24.154449 13141 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.245.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.245.188@KRBTEST.COM" created.
Aug 06 16:22:24 dist-test-slave-t3q3 krb5kdc[12891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033344, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.189@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:22:24.180675 13025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.157764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46782 (local address 127.11.245.190:41333)
0806 16:22:24.157977 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.157983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.159426 (+ 1443us) server_negotiation.cc:407] Connection header received
0806 16:22:24.161492 (+ 2066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.161497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.161581 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.161678 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.163784 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.164655 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.167355 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.167712 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.167948 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.173252 (+ 5304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:24.173274 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:24.173280 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:24.173331 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:24.176795 (+ 3464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.177761 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.177772 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.177774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.177891 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.178614 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.178618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.178620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.178960 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:24.179313 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:24.180229 (+ 916us) server_negotiation.cc:299] Negotiation successful
0806 16:22:24.180453 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 16:22:24.182116 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.158374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:41333 (local address 127.0.0.1:46782)
0806 16:22:24.159089 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.159165 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:24.161140 (+ 1975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:24.162022 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:24.162033 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:24.162737 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:24.163471 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:24.163514 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.165000 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.165006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:24.167066 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:24.167106 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.168104 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.168112 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:24.168509 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.169579 (+ 1070us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:24.169609 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:24.172830 (+ 3221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:24.177065 (+ 4235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:24.177076 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:24.177104 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:24.177596 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:24.178014 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:24.178018 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:24.178020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:24.178465 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:24.179309 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:24.179320 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:24.179827 (+ 507us) client_negotiation.cc:769] Sending connection context
0806 16:22:24.180155 (+ 328us) client_negotiation.cc:240] Negotiation successful
0806 16:22:24.180674 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":192,"threads_started":1}
I20260806 16:22:24.185263 13141 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:22:24 dist-test-slave-t3q3 krb5kdc[12891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033344, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.189@KRBTEST.COM for kudu/127.11.245.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.245.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.188.keytab.
Entry for principal kudu/127.11.245.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.188.keytab.
I20260806 16:22:24.198316 12246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
/tmp/dist-test-taskmdsaCi/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.188:34953
--webserver_interface=127.11.245.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.188.keytab
--principal=kudu/127.11.245.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.245.148:44517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.245.190:41333,127.11.245.189:44055,127.11.245.188:34953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:22:24.202111 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.186260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:44055 (local address 127.0.0.1:45390)
0806 16:22:24.186475 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.186494 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:24.186619 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:24.189423 (+ 2804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:24.189431 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:24.189690 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:24.189841 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:24.189848 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.191500 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.191503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:24.192314 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:24.192326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.193785 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.193789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:24.193907 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.194574 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:24.194589 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:24.196777 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:24.199711 (+ 2934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:24.199718 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:24.199723 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:24.200040 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:24.200528 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:24.200535 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:24.200539 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:24.200661 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:24.201602 (+ 941us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:24.201605 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:24.201705 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 16:22:24.201794 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 16:22:24.201943 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 16:22:24.202440 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.186499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45390 (local address 127.11.245.189:44055)
0806 16:22:24.187178 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.187190 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.187229 (+ 39us) server_negotiation.cc:407] Connection header received
0806 16:22:24.187603 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.187614 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.189047 (+ 1433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.189458 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.190109 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.191120 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.192471 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.193538 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.193726 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.197130 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:24.197160 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:24.197197 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:24.197227 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:24.199543 (+ 2316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.200266 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.200273 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.200275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.200369 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.200784 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.200787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.200807 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.201452 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:24.201639 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:24.201964 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 16:22:24.202182 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":267,"threads_started":1}
I20260806 16:22:24.205499 13141 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:22:24.206935 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.206228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35152)
0806 16:22:24.206479 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.206625 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260806 16:22:24.207284 13058 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
I20260806 16:22:24.208752 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.208437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35162)
0806 16:22:24.208598 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.208671 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 16:22:24.209164 13141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:34953: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
I20260806 16:22:24.221828 13020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } attempt: 4
Aug 06 16:22:24 dist-test-slave-t3q3 krb5kdc[12891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for kudu/127.11.245.189@KRBTEST.COM
I20260806 16:22:24.235909 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.222825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:44055 (local address 127.0.0.1:45400)
0806 16:22:24.223124 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.223145 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:24.223320 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:24.223631 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:24.223634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:24.224046 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:24.224354 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:24.224367 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.225642 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.225646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:24.226368 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:24.226391 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.227868 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.227874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:24.228017 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.228878 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:24.228897 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:24.230681 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:24.233297 (+ 2616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:24.233301 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:24.233304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:24.233612 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:24.234320 (+ 708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:24.234324 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:24.234330 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:24.234616 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:24.235190 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:24.235194 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:24.235328 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 16:22:24.235458 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 16:22:24.235686 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260806 16:22:24.235909 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.222965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45400 (local address 127.11.245.189:44055)
0806 16:22:24.223134 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.223141 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.223234 (+ 93us) server_negotiation.cc:407] Connection header received
0806 16:22:24.223451 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.223455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.223513 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.223610 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.224572 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.225474 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.226599 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.227668 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.227828 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.230930 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:24.230935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:24.230938 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:24.230979 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:24.233093 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.233755 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.233759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.233761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.234158 (+ 397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.234768 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.234773 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.234775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.235065 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:24.235195 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:24.235523 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 16:22:24.235686 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 16:22:24.237082 13020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:22:24.238232 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.237785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35166)
0806 16:22:24.237985 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.238060 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 16:22:24.239081 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.238835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35180)
0806 16:22:24.238958 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.239018 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 16:22:24.239245 13020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:34953: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
I20260806 16:22:24.258585 13141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } attempt: 1
I20260806 16:22:24.259969 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.259468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35194)
0806 16:22:24.259761 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.259839 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173}
I20260806 16:22:24.261071 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.260778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35200)
0806 16:22:24.260933 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.260991 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 16:22:24.261260 13141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:34953: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
I20260806 16:22:24.298502 13020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } attempt: 1
I20260806 16:22:24.299863 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.299436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35204)
0806 16:22:24.299673 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.299751 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260806 16:22:24.300760 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.300512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35208)
0806 16:22:24.300632 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.300690 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 16:22:24.300904 13020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:34953: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
I20260806 16:22:24.329519 13141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } attempt: 2
I20260806 16:22:24.331018 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.330566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35212)
0806 16:22:24.330819 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.330911 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260806 16:22:24.332520 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.332239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35228)
0806 16:22:24.332361 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.332434 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 16:22:24.332711 13141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:34953: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
W20260806 16:22:24.366530 13151 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:22:24.367000 13151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:22:24.367158 13151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:22:24.367259 13151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 16:22:24.368151 13020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } attempt: 2
I20260806 16:22:24.369481 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.369099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35240)
0806 16:22:24.369285 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.369372 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 16:22:24.370612 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.370270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35256)
0806 16:22:24.370442 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.370517 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 16:22:24.370795 13020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:34953: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
W20260806 16:22:24.373334 13151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:22:24.373425 13151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:22:24.373449 13151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:22:24.373469 13151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:22:24.373486 13151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:22:24.381381 13151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.245.148:44517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.245.190:41333,127.11.245.189:44055,127.11.245.188:34953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/krb5kdc/kudu_127.11.245.188.keytab
--principal=kudu/127.11.245.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.245.188:34953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.245.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:22:24.383096 13151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:22:24.384701 13151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:22:24.396196 13165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:24.396391 13161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:22:24.396370 13163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:22:24.396730 13151 server_base.cc:1034] running on GCE node
I20260806 16:22:24.397626 13151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:22:24.399297 13151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:22:24.400529 13151 hybrid_clock.cc:648] HybridClock initialized: now 1786033344400483 us; error 69 us; skew 500 ppm
Aug 06 16:22:24 dist-test-slave-t3q3 krb5kdc[12891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033344, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:22:24.408257 13151 init.cc:377] Logged in from keytab as kudu/127.11.245.188@KRBTEST.COM (short username kudu)
I20260806 16:22:24.409965 13151 webserver.cc:460] Webserver started at http://127.11.245.188:45223/ using document root <none> and password file <none>
I20260806 16:22:24.410820 13151 fs_manager.cc:359] Metadata directory not provided
I20260806 16:22:24.410903 13151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:22:24.411204 13151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:22:24.414371 13151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/data/instance:
uuid: "84fb0f49d5ef4de49b6e821fa097998d"
format_stamp: "Formatted at 2026-08-06 16:22:24 on dist-test-slave-t3q3"
server_key: "132cc4956e7bd97118bce599f1fa3405"
server_key_iv: "20bb1985a8f6810688a406eeb34649ee"
server_key_version: "encryptionkey@0"
I20260806 16:22:24.415228 13151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/wal/instance:
uuid: "84fb0f49d5ef4de49b6e821fa097998d"
format_stamp: "Formatted at 2026-08-06 16:22:24 on dist-test-slave-t3q3"
server_key: "132cc4956e7bd97118bce599f1fa3405"
server_key_iv: "20bb1985a8f6810688a406eeb34649ee"
server_key_version: "encryptionkey@0"
I20260806 16:22:24.421008 13151 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:22:24.423933 13141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } attempt: 3
I20260806 16:22:24.425084 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.424718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35266)
0806 16:22:24.424919 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.424988 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 16:22:24.425312 13176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:22:24.425951 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.425662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35270)
0806 16:22:24.425838 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.425892 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260806 16:22:24.426144 13141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.245.188:34953: Network error: Client connection negotiation failed: client connection to 127.11.245.188:34953: connect: Connection refused (error 111)
I20260806 16:22:24.427353 13151 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:22:24.427543 13151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/data,/tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/wal
uuid: "84fb0f49d5ef4de49b6e821fa097998d"
format_stamp: "Formatted at 2026-08-06 16:22:24 on dist-test-slave-t3q3"
server_key: "132cc4956e7bd97118bce599f1fa3405"
server_key_iv: "20bb1985a8f6810688a406eeb34649ee"
server_key_version: "encryptionkey@0"
I20260806 16:22:24.427711 13151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:22:24.453339 13151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:22:24.457875 13151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:22:24.458159 13151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:22:24.469254 13151 rpc_server.cc:307] RPC server started. Bound to: 127.11.245.188:34953
I20260806 16:22:24.469626 13245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.245.188:34953 every 8 connection(s)
I20260806 16:22:24.470851 13151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/data/info.pb
I20260806 16:22:24.471002 13020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } attempt: 3
I20260806 16:22:24.473474 12246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu as pid 13151
I20260806 16:22:24.473656 12246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmdsaCi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786033011181791-12246-0/minicluster-data/master-2/wal/instance
I20260806 16:22:24.474118 12246 external_mini_cluster.cc:1428] Setting key 3906eebf4451f35b3296cfb3dbd01e2f
I20260806 16:22:24.476272 13250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 16:22:24 dist-test-slave-t3q3 krb5kdc[12891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.190@KRBTEST.COM for kudu/127.11.245.188@KRBTEST.COM
Aug 06 16:22:24 dist-test-slave-t3q3 krb5kdc[12891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.188@KRBTEST.COM
I20260806 16:22:24.483651 13250 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:22:24.489108 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.471783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35282)
0806 16:22:24.471983 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.472005 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:24.472165 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:24.476264 (+ 4099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:24.476269 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:24.476562 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:24.476731 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:24.476741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.478662 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.478667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:24.479273 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:24.479283 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.480385 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.480390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:24.480498 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.481336 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:24.481374 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:24.482965 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:24.486357 (+ 3392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:24.486364 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:24.486369 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:24.486694 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:24.487195 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:24.487198 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:24.487201 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:24.487292 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:24.488414 (+ 1122us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:24.488418 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:24.488579 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 16:22:24.488681 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 16:22:24.488868 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 16:22:24.490661 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.474966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35296 (local address 127.11.245.188:34953)
0806 16:22:24.475406 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.475423 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.475722 (+ 299us) server_negotiation.cc:407] Connection header received
0806 16:22:24.475924 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.475928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.475983 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.476314 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.477801 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.478500 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.480302 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.480672 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.480813 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.484169 (+ 3356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:24.484175 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:24.484178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:24.484215 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:24.486301 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.486864 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.486871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.486879 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.487069 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.487406 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.487409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.487423 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.488276 (+ 853us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:24.488534 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:24.489049 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 16:22:24.489421 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":102,"server-negotiator.queue_time_us":355,"spinlock_wait_cycles":48640,"thread_start_us":185,"threads_started":1}
I20260806 16:22:24.490658 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.473412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35282 (local address 127.11.245.188:34953)
0806 16:22:24.474081 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.474106 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.474164 (+ 58us) server_negotiation.cc:407] Connection header received
0806 16:22:24.475226 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.475257 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.475841 (+ 584us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.476315 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.476935 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.478500 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.479417 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.480243 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.480446 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.483204 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:24.483210 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:24.483230 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:24.483264 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:24.486174 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.486864 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.486871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.486879 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.487069 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.487406 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.487409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.487411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.488276 (+ 865us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:24.489318 (+ 1042us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:24.489408 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 16:22:24.489559 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":121,"server-negotiator.queue_time_us":462,"spinlock_wait_cycles":16384,"thread_start_us":248,"threads_started":1}
Aug 06 16:22:24 dist-test-slave-t3q3 krb5kdc[12891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033344, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.188@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:22:24.496833 13020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8: Bootstrap starting.
I20260806 16:22:24.499408 13256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.486475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.190:41333 (local address 127.0.0.1:46794)
0806 16:22:24.487015 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.487074 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:24.487265 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:24.487667 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:24.487673 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:24.488174 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:24.488481 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:24.488495 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.489530 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.489534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:24.490492 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:24.490504 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.491219 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.491223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:24.491335 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.492781 (+ 1446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:24.492800 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:24.494894 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:24.496962 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:24.496978 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:24.496987 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:24.497383 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:24.497751 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:24.497754 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:24.497757 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:24.497866 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:24.498414 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:24.498423 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:24.498607 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 16:22:24.498810 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 16:22:24.499105 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":160,"threads_started":1}
I20260806 16:22:24.499577 13025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.486517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46794 (local address 127.11.245.190:41333)
0806 16:22:24.486695 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.486699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.487162 (+ 463us) server_negotiation.cc:407] Connection header received
0806 16:22:24.487374 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.487378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.487437 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.487494 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.488635 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.489389 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.490632 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.490968 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.491081 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.495059 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:24.495063 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:24.495065 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:24.495089 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:24.496810 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.497505 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.497510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.497512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.497639 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.497995 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.497998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.498000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.498281 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:24.498402 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:24.498892 (+ 490us) server_negotiation.cc:299] Negotiation successful
0806 16:22:24.499367 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 16:22:24 dist-test-slave-t3q3 krb5kdc[12891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.190@KRBTEST.COM
I20260806 16:22:24.501415 13020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8: Neither blocks nor log segments found. Creating new log.
I20260806 16:22:24.501624 13250 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:22:24.502712 13020 log.cc:824] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8: Log is configured to *not* fsync() on all Append() calls
I20260806 16:22:24.506312 13020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8: No bootstrap required, opened a new log
I20260806 16:22:24.506660 13257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.494555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46798 (local address 127.11.245.190:41333)
0806 16:22:24.494953 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.494969 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.494984 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:22:24.495023 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.495027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.495076 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.496099 (+ 1023us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.496216 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.497026 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.498163 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.498500 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.498614 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.502001 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:24.502005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:24.502008 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:24.502043 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:24.504032 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.504603 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.504607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.504609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.504714 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.505030 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.505033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.505035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.505297 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:24.506218 (+ 921us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:24.506302 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 16:22:24.506466 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":132,"threads_started":1}
Aug 06 16:22:24 dist-test-slave-t3q3 krb5kdc[12891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033344, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.188@KRBTEST.COM for kudu/127.11.245.189@KRBTEST.COM
I20260806 16:22:24.513705 13256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.502331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.189:44055 (local address 127.0.0.1:45406)
0806 16:22:24.502487 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.502504 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:24.502835 (+ 331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:24.503259 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:24.503262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:24.503592 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:24.503753 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:24.503760 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.505051 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.505054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:24.505659 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:24.505669 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.506222 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.506226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:24.506342 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.506939 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:24.506954 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:24.508663 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:24.510994 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:24.510998 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:24.511000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:24.511302 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:24.512197 (+ 895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:24.512201 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:24.512203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:24.512327 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:24.512865 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:24.512871 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:24.513041 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 16:22:24.513176 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 16:22:24.513427 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 16:22:24.513891 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.502701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45406 (local address 127.11.245.189:44055)
0806 16:22:24.502884 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.502890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.502907 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:22:24.502961 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.502964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.503022 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.503108 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.503896 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.504719 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.505799 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.506104 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.506204 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.508821 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:24.508826 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:24.508829 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:24.508860 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:24.510834 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.511914 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.511919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.511921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.512030 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.512442 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.512446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.512448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.512743 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:24.512863 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:24.513264 (+ 401us) server_negotiation.cc:299] Negotiation successful
0806 16:22:24.513428 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":69}
I20260806 16:22:24.513736 13020 raft_consensus.cc:348] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } }
I20260806 16:22:24.514153 13020 raft_consensus.cc:729] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 836458b37909437886c0e17ac02f42e8, State: Initialized, Role: FOLLOWER
I20260806 16:22:24.514714 13020 consensus_queue.cc:260] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } }
I20260806 16:22:24.514904 13250 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:22:24.515575 13262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } } }
I20260806 16:22:24.515746 13262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 16:22:24 dist-test-slave-t3q3 krb5kdc[12891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.245.189@KRBTEST.COM
I20260806 16:22:24.517220 13020 sys_catalog.cc:564] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 16:22:24 dist-test-slave-t3q3 krb5kdc[12891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033344, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.188@KRBTEST.COM for kudu/127.11.245.188@KRBTEST.COM
I20260806 16:22:24.522086 13260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.508113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45418 (local address 127.11.245.189:44055)
0806 16:22:24.508946 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.508954 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.508965 (+ 11us) server_negotiation.cc:407] Connection header received
0806 16:22:24.508995 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.508997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.509035 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.509143 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.509970 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.510600 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.511668 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.512357 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.512559 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.516805 (+ 4246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:24.516814 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:24.516817 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:24.516842 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:24.519115 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.519843 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.519850 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.519853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.519960 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.520368 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.520372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.520374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.520663 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:24.520903 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:24.521703 (+ 800us) server_negotiation.cc:299] Negotiation successful
0806 16:22:24.521873 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":760,"thread_start_us":172,"threads_started":1}
W20260806 16:22:24.525450 13278 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:22:24.525651 13278 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:22:24.526335 13256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.515560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35312)
0806 16:22:24.515894 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.515912 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:24.516005 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:24.516584 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:24.516587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:24.516783 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:24.516913 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:24.516920 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.518104 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.518108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:24.518909 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:24.518920 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.519418 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.519421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:24.519502 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.520017 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:24.520032 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:24.521792 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:24.524429 (+ 2637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:24.524436 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:24.524440 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:24.524736 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:24.525073 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:24.525076 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:24.525078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:24.525148 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:24.525670 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:24.525672 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:24.525747 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 16:22:24.525868 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 16:22:24.526041 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214}
I20260806 16:22:24.526335 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.516173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35312 (local address 127.11.245.188:34953)
0806 16:22:24.516301 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.516307 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.516322 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:22:24.516365 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.516371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.516444 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.516539 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.517065 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.517976 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.519030 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.519307 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.519424 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.522100 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:24.522106 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:24.522111 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:24.522137 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:24.524298 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.524871 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.524877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.524880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.524974 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.525230 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.525233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.525235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.525537 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:24.525638 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:24.525971 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 16:22:24.526128 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 16:22:24.529057 12246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:22:24.530699 12975 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "836458b37909437886c0e17ac02f42e8"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46798
I20260806 16:22:24.531045 12975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 16:22:24.531061 13250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d: Bootstrap starting.
I20260806 16:22:24.531183 12975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:22:24.532946 12975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } }
I20260806 16:22:24.524238 12913 master_runner.cc:186] Error getting master registration for 127.11.245.189:44055: OK, instance_id { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" instance_seqno: 1786033344138139 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:22:24.535272 13250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d: Neither blocks nor log segments found. Creating new log.
I20260806 16:22:24.535454 12913 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:22:24.535844 12975 leader_election.cc:288] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [CANDIDATE]: Term 1 election: Requested vote from peers 1757dee06d4f446daf2e11085b417a6c (127.11.245.189:44055), 84fb0f49d5ef4de49b6e821fa097998d (127.11.245.188:34953)
I20260806 16:22:24.536729 13250 log.cc:824] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d: Log is configured to *not* fsync() on all Append() calls
I20260806 16:22:24.540405 13250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d: No bootstrap required, opened a new log
I20260806 16:22:24.546452 13250 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } }
I20260806 16:22:24.546870 13250 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84fb0f49d5ef4de49b6e821fa097998d, State: Initialized, Role: FOLLOWER
I20260806 16:22:24.547420 13250 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } }
W20260806 16:22:24.552836 12947 tablet.cc:2369] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:22:24.555244 13257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.540268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46806 (local address 127.11.245.190:41333)
0806 16:22:24.540424 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.540431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.541322 (+ 891us) server_negotiation.cc:407] Connection header received
0806 16:22:24.541518 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.541522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.541584 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.541643 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.543510 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.544552 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.546495 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.546909 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.547002 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.550254 (+ 3252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:24.550259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:24.550262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:24.550299 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:24.552334 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.553351 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.553355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.553357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.553472 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.554047 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.554053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.554055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.554333 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:24.554464 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:24.554854 (+ 390us) server_negotiation.cc:299] Negotiation successful
0806 16:22:24.555037 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 16:22:24.557544 13250 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:22:24.557824 13211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "836458b37909437886c0e17ac02f42e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "84fb0f49d5ef4de49b6e821fa097998d"
I20260806 16:22:24.558214 13211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:22:24.560390 13260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.539464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45430 (local address 127.11.245.189:44055)
0806 16:22:24.548409 (+ 8945us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.548416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.548435 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:22:24.548497 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.548500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.548565 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.548670 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.549883 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.550860 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.551998 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.552458 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.552572 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.555230 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:24.555235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:24.555239 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:24.555455 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:24.557778 (+ 2323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.558388 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.558393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.558395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.558501 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.558813 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.558816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.558828 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.559105 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:24.559237 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:24.559993 (+ 756us) server_negotiation.cc:299] Negotiation successful
0806 16:22:24.560191 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8788}
I20260806 16:22:24.560518 13211 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 836458b37909437886c0e17ac02f42e8 in term 1.
I20260806 16:22:24.561259 12941 leader_election.cc:302] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 836458b37909437886c0e17ac02f42e8, 84fb0f49d5ef4de49b6e821fa097998d; no voters:
I20260806 16:22:24.561323 13289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } } }
I20260806 16:22:24.561533 13289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:22:24.561806 13262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:22:24.562122 13262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [term 1 LEADER]: Becoming Leader. State: Replica: 836458b37909437886c0e17ac02f42e8, State: Running, Role: LEADER
I20260806 16:22:24.563138 13262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } }
I20260806 16:22:24.566051 13262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 836458b37909437886c0e17ac02f42e8. Latest consensus state: current_term: 1 leader_uuid: "836458b37909437886c0e17ac02f42e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } } }
I20260806 16:22:24.566259 13262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:24.566560 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.541435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35326 (local address 127.11.245.188:34953)
0806 16:22:24.541565 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.541570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.542650 (+ 1080us) server_negotiation.cc:407] Connection header received
0806 16:22:24.542917 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.542922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.542983 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.543041 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.544738 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.545553 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.547408 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.547732 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.547841 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.553243 (+ 5402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:24.553252 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:24.553257 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:24.553301 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:24.555801 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.564449 (+ 8648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.564469 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.564472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.564606 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.565121 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.565125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.565127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.565414 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:24.565553 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:24.566191 (+ 638us) server_negotiation.cc:299] Negotiation successful
0806 16:22:24.566377 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 16:22:24.566830 13303 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:22:24.566924 13303 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:22:24.569341 13141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } attempt: 4
I20260806 16:22:24.572472 13151 master_runner.cc:186] Error getting master registration for 127.11.245.189:44055: OK, instance_id { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" instance_seqno: 1786033344138139 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:22:24.576987 13151 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 06 16:22:24 dist-test-slave-t3q3 krb5kdc[12891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786033344, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.245.189@KRBTEST.COM for kudu/127.11.245.188@KRBTEST.COM
I20260806 16:22:24.578837 13309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:22:24.580375 13309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:22:24.583765 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.569934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.245.188:34953 (local address 127.0.0.1:35334)
0806 16:22:24.570164 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 16:22:24.570180 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:22:24.570269 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:22:24.570551 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:22:24.570553 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:22:24.570788 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:22:24.570923 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:24.570931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.571842 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.571846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:24.572802 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:22:24.572814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.576350 (+ 3536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.576355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:22:24.576478 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.577240 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:22:24.577256 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:22:24.579061 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:22:24.581687 (+ 2626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:24.581691 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:24.581694 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:24.581969 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:24.582363 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:22:24.582366 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:22:24.582368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:22:24.582488 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:22:24.583089 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:22:24.583092 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:22:24.583228 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 16:22:24.583345 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 16:22:24.583563 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260806 16:22:24.584481 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:24.570001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35334 (local address 127.11.245.188:34953)
0806 16:22:24.570223 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.570228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.570243 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:22:24.570371 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.570375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.570428 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.570514 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.571035 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.571748 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.572919 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:24.573268 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:24.573380 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:24.579264 (+ 5884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:22:24.579268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:22:24.579271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:22:24.579304 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:22:24.581492 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.582116 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.582134 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.582136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.582233 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:22:24.582619 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:22:24.582622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:22:24.582624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:22:24.582928 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:22:24.583027 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:24.584164 (+ 1137us) server_negotiation.cc:299] Negotiation successful
0806 16:22:24.584312 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260806 16:22:24.588490 13211 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d [term 1 FOLLOWER]: Refusing update from remote peer 836458b37909437886c0e17ac02f42e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:22:24.589506 13141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c: Bootstrap starting.
I20260806 16:22:24.589587 13304 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:22:24.593703 13141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c: Neither blocks nor log segments found. Creating new log.
I20260806 16:22:24.594936 13141 log.cc:824] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c: Log is configured to *not* fsync() on all Append() calls
I20260806 16:22:24.597157 13289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 836458b37909437886c0e17ac02f42e8. Latest consensus state: current_term: 1 leader_uuid: "836458b37909437886c0e17ac02f42e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } } }
I20260806 16:22:24.597474 13289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:22:24.601670 13262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "836458b37909437886c0e17ac02f42e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } } }
I20260806 16:22:24.601871 13262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [sys.catalog]: This master's current role is: LEADER
I20260806 16:22:24.602413 13289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "836458b37909437886c0e17ac02f42e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } } }
I20260806 16:22:24.602573 13289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:22:24.602972 13310 mvcc.cc:205] Tried to move back new op lower bound from 7315592579423465472 to 7315592579336896512. Current Snapshot: MvccSnapshot[applied={T|T < 7315592579423465472}]
I20260806 16:22:24.606045 13141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c: No bootstrap required, opened a new log
I20260806 16:22:24.606621 13311 mvcc.cc:205] Tried to move back new op lower bound from 7315592579423465472 to 7315592579336896512. Current Snapshot: MvccSnapshot[applied={T|T < 7315592579423465472}]
I20260806 16:22:24.609311 13309 catalog_manager.cc:1347] Generated new cluster ID: 5bf749db67c14844b60f5487d9a13931
I20260806 16:22:24.609413 13309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:22:24.615366 13141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } }
I20260806 16:22:24.616264 13141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1757dee06d4f446daf2e11085b417a6c, State: Initialized, Role: FOLLOWER
I20260806 16:22:24.617055 13141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } }
I20260806 16:22:24.621332 13098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:22:24.624022 13098 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c [term 1 FOLLOWER]: Refusing update from remote peer 836458b37909437886c0e17ac02f42e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:22:24.624616 13099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "836458b37909437886c0e17ac02f42e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1757dee06d4f446daf2e11085b417a6c"
I20260806 16:22:24.625572 13304 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:22:24.626137 13099 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 836458b37909437886c0e17ac02f42e8 in term 1.
I20260806 16:22:24.626909 13141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c [sys.catalog]: configured and running, proceeding with master startup.
W20260806 16:22:24.632309 13065 tablet.cc:2369] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:22:24.634830 13337 mvcc.cc:205] Tried to move back new op lower bound from 7315592579423465472 to 7315592579336896512. Current Snapshot: MvccSnapshot[applied={T|T < 7315592579423465472}]
I20260806 16:22:24.638972 13317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 836458b37909437886c0e17ac02f42e8. Latest consensus state: current_term: 1 leader_uuid: "836458b37909437886c0e17ac02f42e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } } }
I20260806 16:22:24.639285 13317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:22:24.639478 13317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "836458b37909437886c0e17ac02f42e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } } }
I20260806 16:22:24.639649 13317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:22:24.645565 13262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "836458b37909437886c0e17ac02f42e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } } }
I20260806 16:22:24.645794 13262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8 [sys.catalog]: This master's current role is: LEADER
W20260806 16:22:24.645753 13345 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:22:24.646183 13345 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:22:24.646417 13309 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:22:24.648116 13309 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:22:24.637259 13343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "836458b37909437886c0e17ac02f42e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "836458b37909437886c0e17ac02f42e8" member_type: VOTER last_known_addr { host: "127.11.245.190" port: 41333 } } peers { permanent_uuid: "1757dee06d4f446daf2e11085b417a6c" member_type: VOTER last_known_addr { host: "127.11.245.189" port: 44055 } } peers { permanent_uuid: "84fb0f49d5ef4de49b6e821fa097998d" member_type: VOTER last_known_addr { host: "127.11.245.188" port: 34953 } } }
I20260806 16:22:24.655596 13343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:22:24.676206 13309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 836458b37909437886c0e17ac02f42e8: Generated new TSK 0
I20260806 16:22:24.677489 13309 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 16:22:24.724910 13260 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:24.720642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45446 (local address 127.11.245.189:44055)
0806 16:22:24.720814 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.720821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.723242 (+ 2421us) server_negotiation.cc:407] Connection header received
0806 16:22:24.723542 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.723548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.723621 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.723690 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.724684 (+ 994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45446: BlockingRecv error: recv got EOF from 127.0.0.1:45446 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 16:22:24.727002 13254 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:24.722456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35350 (local address 127.11.245.188:34953)
0806 16:22:24.722609 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.722615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.725644 (+ 3029us) server_negotiation.cc:407] Connection header received
0806 16:22:24.725941 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.725945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.726019 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.726082 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.726828 (+ 746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35350: BlockingRecv error: recv got EOF from 127.0.0.1:35350 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 16:22:24.733147 13257 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:24.721689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46818 (local address 127.11.245.190:41333)
0806 16:22:24.721839 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.721844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.725518 (+ 3674us) server_negotiation.cc:407] Connection header received
0806 16:22:24.727662 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.727667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.727743 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.727830 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.732908 (+ 5078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46818: BlockingRecv error: recv got EOF from 127.0.0.1:46818 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 16:22:24.758140 13260 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:24.744884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45462 (local address 127.11.245.189:44055)
0806 16:22:24.745061 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.745068 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.756444 (+ 11376us) server_negotiation.cc:407] Connection header received
0806 16:22:24.756773 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.756777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.756904 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.757015 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.757974 (+ 959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45462: BlockingRecv error: recv got EOF from 127.0.0.1:45462 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 16:22:24.773468 13260 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:24.762283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45464 (local address 127.11.245.189:44055)
0806 16:22:24.762503 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.762508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.771429 (+ 8921us) server_negotiation.cc:407] Connection header received
0806 16:22:24.771552 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.771556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.771648 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.771764 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.773291 (+ 1527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45464: BlockingRecv error: recv got EOF from 127.0.0.1:45464 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 16:22:24.794210 13260 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:24.781281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45472 (local address 127.11.245.189:44055)
0806 16:22:24.781463 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.781470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.792701 (+ 11231us) server_negotiation.cc:407] Connection header received
0806 16:22:24.793100 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.793105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.793187 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.793255 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.794040 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45472: BlockingRecv error: recv got EOF from 127.0.0.1:45472 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 16:22:24.810119 13260 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:24.803578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45488 (local address 127.11.245.189:44055)
0806 16:22:24.803786 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.803792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.808706 (+ 4914us) server_negotiation.cc:407] Connection header received
0806 16:22:24.809062 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.809067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.809155 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.809288 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.809941 (+ 653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45488: BlockingRecv error: recv got EOF from 127.0.0.1:45488 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 16:22:24.823509 13260 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:24.821297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45492 (local address 127.11.245.189:44055)
0806 16:22:24.821494 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.821501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.822118 (+ 617us) server_negotiation.cc:407] Connection header received
0806 16:22:24.822410 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.822414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.822490 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.822561 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.823331 (+ 770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45492: BlockingRecv error: recv got EOF from 127.0.0.1:45492 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 16:22:24.844944 13260 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:24.843239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45498 (local address 127.11.245.189:44055)
0806 16:22:24.843400 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.843407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.843734 (+ 327us) server_negotiation.cc:407] Connection header received
0806 16:22:24.843922 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.843926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.844006 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.844160 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.844772 (+ 612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45498: BlockingRecv error: recv got EOF from 127.0.0.1:45498 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 16:22:24.881673 13260 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:24.879807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45508 (local address 127.11.245.189:44055)
0806 16:22:24.880013 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.880020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.880375 (+ 355us) server_negotiation.cc:407] Connection header received
0806 16:22:24.880541 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.880545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.880628 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.880748 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.881506 (+ 758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45508: BlockingRecv error: recv got EOF from 127.0.0.1:45508 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 16:22:24.950382 13260 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:24.948888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45516 (local address 127.11.245.189:44055)
0806 16:22:24.949087 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:24.949097 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:24.949228 (+ 131us) server_negotiation.cc:407] Connection header received
0806 16:22:24.949467 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:24.949471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:24.949530 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:24.949626 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:24.950206 (+ 580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45516: BlockingRecv error: recv got EOF from 127.0.0.1:45516 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260806 16:22:25.083050 13260 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:25.081473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45522 (local address 127.11.245.189:44055)
0806 16:22:25.081729 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:25.081736 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:25.081909 (+ 173us) server_negotiation.cc:407] Connection header received
0806 16:22:25.082135 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:25.082138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:25.082197 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:25.082296 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:25.082869 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45522: BlockingRecv error: recv got EOF from 127.0.0.1:45522 (error 108)
Metrics: {"server-negotiator.queue_time_us":130}
W20260806 16:22:25.343699 13260 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:22:25.342382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45526 (local address 127.11.245.189:44055)
0806 16:22:25.342569 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:25.342573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:25.342593 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:22:25.342766 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:25.342769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:25.342827 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:25.342927 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:22:25.343570 (+ 643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45526: BlockingRecv error: recv got EOF from 127.0.0.1:45526 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 16:22:25.568928 13303 catalog_manager.cc:1259] Loaded cluster ID: 5bf749db67c14844b60f5487d9a13931
I20260806 16:22:25.569104 13303 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d: loading cluster ID for follower catalog manager: success
I20260806 16:22:25.571240 13303 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d: acquiring CA information for follower catalog manager: success
I20260806 16:22:25.572959 13303 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 84fb0f49d5ef4de49b6e821fa097998d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:22:25.649394 13345 catalog_manager.cc:1259] Loaded cluster ID: 5bf749db67c14844b60f5487d9a13931
I20260806 16:22:25.649539 13345 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c: loading cluster ID for follower catalog manager: success
I20260806 16:22:25.651822 13345 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c: acquiring CA information for follower catalog manager: success
I20260806 16:22:25.654078 13345 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1757dee06d4f446daf2e11085b417a6c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:22:25.863296 13450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:25.858632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45530 (local address 127.11.245.189:44055)
0806 16:22:25.858969 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:25.858974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:25.858993 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:22:25.859163 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:25.859170 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:25.859358 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:25.859481 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:22:25.859752 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:25.860358 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:25.861371 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:25.861611 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:25.861696 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:25.862105 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:22:25.862807 (+ 702us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:22:25.862915 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:25.863061 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 16:22:25.863130 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":113,"threads_started":1}
I20260806 16:22:25.871721 13457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:25.867558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35358 (local address 127.11.245.188:34953)
0806 16:22:25.867841 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:25.867846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:25.867861 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:22:25.868007 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:25.868010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:25.868237 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:25.868362 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:22:25.868605 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:25.869190 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:25.870044 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:25.870290 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:25.870374 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:25.870673 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:22:25.871303 (+ 630us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:22:25.871412 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:25.871481 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 16:22:25.871558 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":108,"threads_started":1}
I20260806 16:22:25.880887 13450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:25.876139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45536 (local address 127.11.245.189:44055)
0806 16:22:25.876328 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:25.876335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:25.876592 (+ 257us) server_negotiation.cc:407] Connection header received
0806 16:22:25.876787 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:25.876790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:25.876848 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:25.876959 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:22:25.877280 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:25.878131 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:25.879245 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:25.879623 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:25.879747 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:25.880216 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:22:25.880339 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:22:25.880393 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:25.880601 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 16:22:25.880678 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 16:22:25.881707 13457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:25.876878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35372 (local address 127.11.245.188:34953)
0806 16:22:25.877335 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:25.877340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:25.877427 (+ 87us) server_negotiation.cc:407] Connection header received
0806 16:22:25.877599 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:25.877602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:25.877664 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:25.877751 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:22:25.878034 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:25.878813 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:25.880169 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:25.880504 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:25.880636 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:25.881021 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:22:25.881115 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:22:25.881265 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:25.881458 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 16:22:25.881554 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355}
I20260806 16:22:25.881867 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:22:25.876157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46830 (local address 127.11.245.190:41333)
0806 16:22:25.876473 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0806 16:22:25.876480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:22:25.876744 (+ 264us) server_negotiation.cc:407] Connection header received
0806 16:22:25.877038 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:22:25.877042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:22:25.877311 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:22:25.877478 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:22:25.877772 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:25.878628 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:25.879684 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:22:25.880135 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:22:25.880266 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:22:25.880705 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:22:25.881340 (+ 635us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:22:25.881476 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 16:22:25.881590 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 16:22:25.881670 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":95,"threads_started":1}
I20260806 16:22:25.887970 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 12913
I20260806 16:22:25.909514 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 13031
I20260806 16:22:25.933408 12246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmdsaCi/build/debug/bin/kudu with pid 13151
2026-08-06T16:22:25Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3915 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (7827 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (334754 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST